You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "nodece (via GitHub)" <gi...@apache.org> on 2024/12/23 08:01:07 UTC

[PR] [improve][pip] PIP-398: Subscription replication on the namespace and topic levels [pulsar]

nodece opened a new pull request, #23770:
URL: https://github.com/apache/pulsar/pull/23770

   ### Motivation
   
   Enhance the subscription replication.
   
   ### Documentation
   
   <!-- DO NOT REMOVE THIS SECTION. CHECK THE PROPER BOX ONLY. -->
   
   - [ ] `doc` <!-- Your PR contains doc changes. -->
   - [ ] `doc-required` <!-- Your PR changes impact docs and you will update later -->
   - [x] `doc-not-needed` <!-- Your PR changes do not impact docs -->
   - [ ] `doc-complete` <!-- Docs have been already added -->


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the broker, namespace and topic levels [pulsar]

Posted by "nodece (via GitHub)" <gi...@apache.org>.
nodece commented on PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#issuecomment-2595188015

   > > @yyj8 This PIP has been updated, could you have a chance to review this PIP?
   > 
   > Are you referring to updating the code and modifying the functional design specifications in this pip [pip/pip-398.md](https://github.com/apache/pulsar/pull/23770/files#diff-67fc7a48cc071911c2239d1c628335d4147f54345051f36efd2f18dbcc8339c6) design documentation?
   
   Just for this PIP.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the broker, namespace and topic levels [pulsar]

Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on code in PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#discussion_r1965110367


##########
pip/pip-398.md:
##########
@@ -0,0 +1,245 @@
+# PIP-398: Subscription replication on the broker, namespace and topic levels
+
+# Background knowledge
+
+https://github.com/apache/pulsar/pull/4299 introduces the subscription replication feature on the consumer level:
+
+```java
+Consumer<byte[]> consumer = pulsarClient.newConsumer().topic("topic").replicateSubscriptionState(false/*true*/)
+        .subscriptionName("sub").subscribe();
+```
+
+While this provides flexibility, it introduces overhead in managing replication for a large number of consumers. Users
+need to manually enable the `replicateSubscriptionState` flag for each consumer, which can become cumbersome in
+large-scale deployments.
+
+# Motivation
+
+The key motivation behind this PIP is to simplify subscription replication configuration, especially in failover
+scenarios. When a main cluster goes down and a backup cluster is activated, ensuring that subscription states are
+consistently replicated across clusters is critical for failover scenarios. By extending the replication configuration
+to the broker, namespace and topic levels, the system reduces the need for explicit consumer-level configuration.
+
+# Goals
+
+## In Scope
+
+The PIP aims to provide management of subscription replication at the broker, namespace and topic levels using the
+Pulsar Admin CLI and API.
+
+# High Level Design
+
+Introduces the `replicateSubscriptionState` configuration to enabling subscription replication on the broker, namespace
+and topic levels, when enabled, all consumers under the broker/namespace/topic will automatically replicate their
+subscription states to remote clusters.
+
+The priority for the subscription replication configuration is as follows:
+
+- consumer level > topic level > namespace level > broker level.
+- If `replicateSubscriptionState` is set at the consumer level, configurations at the topic, namespace, and broker levels are
+  ignored.
+- If set at the topic level, the namespace-level configuration is ignored.
+- If set at the namespace level, the broker-level configuration is ignored.
+
+# Detailed Design
+
+## Design & Implementation Details
+
+### Broker level
+
+Add the field `Boolean replicate_subscriptions_state` to the `org.apache.pulsar.broker.ServiceConfiguration` class
+to control subscription replication at the broker level:
+```java
+public class ServiceConfiguration implements PulsarConfiguration {
+    @FieldContext(
+            category = CATEGORY_SERVER,
+            required = false,
+            dynamic = true,
+            doc = "The default value for replicating subscription state."
+    )
+    private Boolean replicate_subscriptions_state;
+}
+```
+
+### Namespace level
+
+1. Add the field `Boolean replicate_subscriptions_state` to the `org.apache.pulsar.common.policies.data.Policies` class
+   to control subscription replication at the namespace level:
+    ```java
+    public class Policies {
+        @SuppressWarnings("checkstyle:MemberName")
+        public Boolean replicate_subscription_state;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.Namespaces` interface:
+     ```java
+     public interface Namespaces {
+         void setReplicateSubscriptionState(String namespace, Boolean enabled) throws PulsarAdminException;
+         CompletableFuture<Void> setReplicateSubscriptionStateAsync(String namespace, Boolean enabled);
+         Boolean getReplicateSubscriptionState(String namespace) throws PulsarAdminException;
+         CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String namespace);
+     }
+     ```
+3. Implement the management methods in the `org.apache.pulsar.client.admin.internal.NamespacesImpl` class.
+
+### Topic level
+
+1. Add the field `Boolean replicateSubscriptionState` to the `org.apache.pulsar.common.policies.data.TopicPolicies`
+   class to enable subscription replication at the topic level:
+   ```java
+    public class TopicPolicies {
+         public Boolean replicateSubscriptionState;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.TopicPolicies` interface:
+   ```java
+       public interface TopicPolicies { 
+           void setReplicateSubscriptionState(String topic, Boolean enabled) throws PulsarAdminException;
+           CompletableFuture<Void> setReplicateSubscriptionStateAsync(String topic, Boolean enabled);
+           Boolean getReplicateSubscriptionState(String topic, boolean applied) throws PulsarAdminException;
+           CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String topic, boolean applied);
+       }
+   ```
+3. Implement the management methods in the `org.apache.pulsar.client.admin.internal.TopicPoliciesImpl` class.
+
+### Consumer level
+
+No changes. When the consumer with `replicateSubscriptionState=true`, the old/new subscription will be snapshot. If `false`, no operation will
+be performed.
+
+### Subscription replication applied
+
+When the subscription replication is changed on the broker, namespace or topic level, the subscription replication will
+be applied to all consumers under the broker/namespace/topic.
+
+We exclusively consider the `true` case for replication. If the consumer level is set to `false`, the
+replication configuration defined at the topic, namespace, or broker level will be applied to the subscription.
+
+There is a special case here, if the user intentionally sets `false` at the consumer level, and then set `true` at the
+topic/ns/broker level, this may disrupt the user's behavior. This way, we can minimize changes to the Pulsar public API
+as much as possible.

Review Comment:
   It would be good to explain what "disrupt the user's behavior" means. Perhaps rewording with the help of Generative AI could be useful since "disruption" is typically used in a different context.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the namespace and topic levels [pulsar]

Posted by "nodece (via GitHub)" <gi...@apache.org>.
nodece commented on PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#issuecomment-2559130884

   > @nodece would it be possible to cover #23769 feature too? That enables subscription replication for all subscriptions at the broker level if I understand it correctly. Could you work together with @yyj8 to cover that?
   
   #23769 aims to overwrite the subscription replication state from the consumer, which is different from this PIP. Actually, I also need an overwrite feature,  and then the users use the admin API to manage the subscription replication.
   
   The downstream users sometimes enable/disable the subscription replication on the consumer, this lead to the namespace or topic level is ignored.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the broker, namespace and topic levels [pulsar]

Posted by "nodece (via GitHub)" <gi...@apache.org>.
nodece commented on PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#issuecomment-2665474538

   > The term `replicateSubscriptionState` is ambiguous (it's not introduced in this PIP). It's unclear whether it refers to "replicating the subscription state" or represents "the state of replicating subscriptions". This demonstrates why putting more thought into naming concepts could be helpful.
   
   "replicating the subscription state" is correct. This term is from the design of existing consumer.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the broker, namespace and topic levels [pulsar]

Posted by "nodece (via GitHub)" <gi...@apache.org>.
nodece commented on PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#issuecomment-4037776994

   @lhotari This PIP has been updated.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the namespace and topic levels [pulsar]

Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#issuecomment-2559143367

   > > @nodece would it be possible to cover #23769 feature too? That enables subscription replication for all subscriptions at the broker level if I understand it correctly. Could you work together with @yyj8 to cover that?
   > 
   > #23769 aims to overwrite the subscription replication state from the consumer, which is different from this PIP. Actually, I also need an overwrite feature, and then the users use the admin API to manage the subscription replication.
   > 
   > The downstream users sometimes enable/disable the subscription replication on the consumer, this lead to the namespace or topic level is ignored.
   > 
   > We can also merge #23769 to this PIP. What's your opinion? @lhotari @yyj8
   
   @nodece @yyj8 Please work together to combine both cases since they are both valuable and useful. I think it would make
   sense to combine the work in a single PIP to ensure that we achieve consistency across the changes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the namespace and topic levels [pulsar]

Posted by "yyj8 (via GitHub)" <gi...@apache.org>.
yyj8 commented on PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#issuecomment-2560530864

   > @yyj8
   > 
   > > cluster dimension, namespace dimension, and topic dimension.
   > 
   > Once the consumer level is configured, it is the highest priority, the cluster, namespace, and topic levels will be ignored, please see [#23769 (comment)](https://github.com/apache/pulsar/pull/23769#issuecomment-2559221226) for details.
   > 
   > > (2)PR #23769 add the function of dynamically configuring parameter `replicateAllSubscriptionState` values through the command `bin/pulse admin brokers update-dynamic-config  --config replicateAllSubscriptionState --value true/false`
   > 
   > This is feasible.
   
   The default configuration for the client is `replicateSubscriptionState=false`. In the scenario of cluster migration, we hope that there is no need for business code modification, and the server will control the replication of the state of all subscriptions in the cluster uniformly. If the configuration priority of the client is the highest, and the client does not display the configuration `replicateSubscriptionState=true`, this goes against our original intention of not requiring business code modification.
   
   My suggestion is to configure the cluster dimension. If `repliceAllSubscriptState=true` is enabled, it will be a mandatory configuration to overwrite the client. If `repliceAllSubscriptState=false`, then use the client's configuration.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the broker, namespace and topic levels [pulsar]

Posted by "nodece (via GitHub)" <gi...@apache.org>.
nodece commented on PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#issuecomment-2674015186

   > Isn't there a need to override any client application consumer level setting with topic or namespace level policies? 
   
   When the topic/namespace level is true, the consumer level will be overried.
   
   > What if Pulsar administrator would explicitly want to disallow or force using replicated subscriptions for a particular topic or namespace while geo-replication is enabled?
   
   The administrator is permitted to enable the subscription replication and subsequently disable it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the broker, namespace and topic levels [pulsar]

Posted by "nodece (via GitHub)" <gi...@apache.org>.
nodece commented on PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#issuecomment-3219585705

   @lhotari Vote has been started.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the broker, namespace and topic levels [pulsar]

Posted by "nodece (via GitHub)" <gi...@apache.org>.
nodece commented on PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#issuecomment-2673719841

   @lhotari Do you have more suggestions? If not, I will send a vote to the mailing list.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the broker, namespace and topic levels [pulsar]

Posted by "nodece (via GitHub)" <gi...@apache.org>.
nodece commented on PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#issuecomment-2591627133

   @yyj8 This PIP has been updated, could you have a chance to review this PIP?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the broker, namespace and topic levels [pulsar]

Posted by "nodece (via GitHub)" <gi...@apache.org>.
nodece commented on code in PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#discussion_r2298445009


##########
pip/pip-398.md:
##########
@@ -0,0 +1,253 @@
+# PIP-398: Subscription replication on the broker, namespace and topic levels
+
+# Background knowledge
+
+https://github.com/apache/pulsar/pull/4299 introduces the subscription replication feature on the consumer level:
+
+```java
+Consumer<byte[]> consumer = pulsarClient.newConsumer().topic("topic").replicateSubscriptionState(false/*true*/)
+        .subscriptionName("sub").subscribe();
+```
+
+While this provides flexibility, it introduces overhead in managing replication for a large number of consumers. Users
+need to manually enable the `replicateSubscriptionState` flag for each consumer, which can become cumbersome in
+large-scale deployments.
+
+# Motivation
+
+The key motivation behind this PIP is to simplify subscription replication configuration, especially in failover
+scenarios. When a main cluster goes down and a backup cluster is activated, ensuring that subscription states are
+consistently replicated across clusters is critical for failover scenarios. By extending the replication configuration
+to the broker, namespace and topic levels, the system reduces the need for explicit consumer-level configuration.
+
+# Goals
+
+## In Scope
+
+The PIP aims to provide management of subscription replication at the broker, namespace and topic levels using the
+Pulsar Admin CLI and API.
+
+# High Level Design
+
+Introduces the `replicateSubscriptionState` configuration to enabling subscription replication on the broker, namespace
+and topic levels, when enabled, all consumers under the broker/namespace/topic will automatically replicate their
+subscription states to remote clusters.
+
+The priority for the subscription replication configuration is as follows:
+
+- consumer level > topic level > namespace level > broker level.
+- If `replicateSubscriptionState` is set at the consumer level, configurations at the topic, namespace, and broker levels are
+  ignored.
+- If set at the topic level, the namespace-level configuration is ignored.
+- If set at the namespace level, the broker-level configuration is ignored.
+
+# Detailed Design
+
+## Design & Implementation Details
+
+### Broker level
+
+Add the field `Boolean replicate_subscriptions_state` to the `org.apache.pulsar.broker.ServiceConfiguration` class
+to control subscription replication at the broker level:
+```java
+public class ServiceConfiguration implements PulsarConfiguration {
+    @FieldContext(
+            category = CATEGORY_SERVER,
+            required = false,
+            dynamic = true,
+            doc = "The default value for replicating subscription state."
+    )
+    private Boolean replicate_subscriptions_state;
+}
+```
+
+### Namespace level
+
+1. Add the field `Boolean replicate_subscriptions_state` to the `org.apache.pulsar.common.policies.data.Policies` class
+   to control subscription replication at the namespace level:
+    ```java
+    public class Policies {
+        @SuppressWarnings("checkstyle:MemberName")
+        public Boolean replicate_subscription_state;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.Namespaces` interface:
+     ```java
+     public interface Namespaces {
+         void setReplicateSubscriptionState(String namespace, Boolean enabled) throws PulsarAdminException;
+         CompletableFuture<Void> setReplicateSubscriptionStateAsync(String namespace, Boolean enabled);
+         Boolean getReplicateSubscriptionState(String namespace) throws PulsarAdminException;
+         CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String namespace);
+     }
+     ```
+3. Implement the management methods in the `org.apache.pulsar.client.admin.internal.NamespacesImpl` class.
+
+### Topic level
+
+1. Add the field `Boolean replicateSubscriptionState` to the `org.apache.pulsar.common.policies.data.TopicPolicies`
+   class to enable subscription replication at the topic level:
+   ```java
+    public class TopicPolicies {
+         public Boolean replicateSubscriptionState;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.TopicPolicies` interface:
+   ```java
+       public interface TopicPolicies { 
+           void setReplicateSubscriptionState(String topic, Boolean enabled) throws PulsarAdminException;
+           CompletableFuture<Void> setReplicateSubscriptionStateAsync(String topic, Boolean enabled);
+           Boolean getReplicateSubscriptionState(String topic, boolean applied) throws PulsarAdminException;
+           CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String topic, boolean applied);
+       }

Review Comment:
   We have a `pulsar-admin topics set-replicated-subscription-status` command, the current implementation uses Boolean for similar that, so I followed the same design for consistency and to avoid introducing a new pattern in the CLI/API.
   
   The meaning is:
   - null → remove configuration(I can add a remove api instead of this, and then use boolean instead of Boolean in set API)
   - true → enable
   - false → disable (**but true has priority if both are set in the different levels**)
   
   Using an enum would make the meaning explicit, but it introduces additional complexity.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the broker, namespace and topic levels [pulsar]

Posted by "nodece (via GitHub)" <gi...@apache.org>.
nodece commented on code in PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#discussion_r2307837323


##########
pip/pip-398.md:
##########
@@ -0,0 +1,253 @@
+# PIP-398: Subscription replication on the broker, namespace and topic levels
+
+# Background knowledge
+
+https://github.com/apache/pulsar/pull/4299 introduces the subscription replication feature on the consumer level:
+
+```java
+Consumer<byte[]> consumer = pulsarClient.newConsumer().topic("topic").replicateSubscriptionState(false/*true*/)
+        .subscriptionName("sub").subscribe();
+```
+
+While this provides flexibility, it introduces overhead in managing replication for a large number of consumers. Users
+need to manually enable the `replicateSubscriptionState` flag for each consumer, which can become cumbersome in
+large-scale deployments.
+
+# Motivation
+
+The key motivation behind this PIP is to simplify subscription replication configuration, especially in failover
+scenarios. When a main cluster goes down and a backup cluster is activated, ensuring that subscription states are
+consistently replicated across clusters is critical for failover scenarios. By extending the replication configuration
+to the broker, namespace and topic levels, the system reduces the need for explicit consumer-level configuration.
+
+# Goals
+
+## In Scope
+
+The PIP aims to provide management of subscription replication at the broker, namespace and topic levels using the
+Pulsar Admin CLI and API.
+
+# High Level Design
+
+Introduces the `replicateSubscriptionState` configuration to enabling subscription replication on the broker, namespace
+and topic levels, when enabled, all consumers under the broker/namespace/topic will automatically replicate their
+subscription states to remote clusters.
+
+The priority for the subscription replication configuration is as follows:
+
+- consumer level > topic level > namespace level > broker level.
+- If `replicateSubscriptionState` is set at the consumer level, configurations at the topic, namespace, and broker levels are
+  ignored.
+- If set at the topic level, the namespace-level configuration is ignored.
+- If set at the namespace level, the broker-level configuration is ignored.
+
+# Detailed Design
+
+## Design & Implementation Details
+
+### Broker level
+
+Add the field `Boolean replicate_subscriptions_state` to the `org.apache.pulsar.broker.ServiceConfiguration` class
+to control subscription replication at the broker level:
+```java
+public class ServiceConfiguration implements PulsarConfiguration {
+    @FieldContext(
+            category = CATEGORY_SERVER,
+            required = false,
+            dynamic = true,
+            doc = "The default value for replicating subscription state."
+    )
+    private Boolean replicate_subscriptions_state;
+}
+```
+
+### Namespace level
+
+1. Add the field `Boolean replicate_subscriptions_state` to the `org.apache.pulsar.common.policies.data.Policies` class
+   to control subscription replication at the namespace level:
+    ```java
+    public class Policies {
+        @SuppressWarnings("checkstyle:MemberName")
+        public Boolean replicate_subscription_state;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.Namespaces` interface:
+     ```java
+     public interface Namespaces {
+         void setReplicateSubscriptionState(String namespace, Boolean enabled) throws PulsarAdminException;
+         CompletableFuture<Void> setReplicateSubscriptionStateAsync(String namespace, Boolean enabled);
+         Boolean getReplicateSubscriptionState(String namespace) throws PulsarAdminException;
+         CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String namespace);
+     }
+     ```
+3. Implement the management methods in the `org.apache.pulsar.client.admin.internal.NamespacesImpl` class.
+
+### Topic level
+
+1. Add the field `Boolean replicateSubscriptionState` to the `org.apache.pulsar.common.policies.data.TopicPolicies`
+   class to enable subscription replication at the topic level:
+   ```java
+    public class TopicPolicies {
+         public Boolean replicateSubscriptionState;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.TopicPolicies` interface:
+   ```java
+       public interface TopicPolicies { 
+           void setReplicateSubscriptionState(String topic, Boolean enabled) throws PulsarAdminException;
+           CompletableFuture<Void> setReplicateSubscriptionStateAsync(String topic, Boolean enabled);
+           Boolean getReplicateSubscriptionState(String topic, boolean applied) throws PulsarAdminException;
+           CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String topic, boolean applied);
+       }

Review Comment:
   > @nodece With a single `Boolean` value, how would you distinguish between the 2 different cases (1. and 3. above) where subscription replication is enabled?
   
   > override any setting by the consumer (or lower level policy) and enable subscription replication for all subscriptions
   
   Subscription replication is enabled if:
   - The consumer sets true
   - The first non-null policy found in the following order is true: topic → namespace → broker. Once a policy is found (even if false), lower levels are ignored.
   
   > no setting at all (higher level setting would take priority). If all policies (broker, ns, topic) are "empty", the consumer setting would get used directly.
   
   Correct.
   
   I understand the concern about using `null` for removing the setting. To make the semantics clearer, I'll add an explicit `removeReplicateSubscriptionState()` API instead of relying on `null`.
   So no need to worry about `Boolean` vs `boolean`, the new API will handle the unset case explicitly.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the broker, namespace and topic levels [pulsar]

Posted by "nodece (via GitHub)" <gi...@apache.org>.
nodece commented on code in PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#discussion_r2304695804


##########
pip/pip-398.md:
##########
@@ -0,0 +1,253 @@
+# PIP-398: Subscription replication on the broker, namespace and topic levels
+
+# Background knowledge
+
+https://github.com/apache/pulsar/pull/4299 introduces the subscription replication feature on the consumer level:
+
+```java
+Consumer<byte[]> consumer = pulsarClient.newConsumer().topic("topic").replicateSubscriptionState(false/*true*/)
+        .subscriptionName("sub").subscribe();
+```
+
+While this provides flexibility, it introduces overhead in managing replication for a large number of consumers. Users
+need to manually enable the `replicateSubscriptionState` flag for each consumer, which can become cumbersome in
+large-scale deployments.
+
+# Motivation
+
+The key motivation behind this PIP is to simplify subscription replication configuration, especially in failover
+scenarios. When a main cluster goes down and a backup cluster is activated, ensuring that subscription states are
+consistently replicated across clusters is critical for failover scenarios. By extending the replication configuration
+to the broker, namespace and topic levels, the system reduces the need for explicit consumer-level configuration.
+
+# Goals
+
+## In Scope
+
+The PIP aims to provide management of subscription replication at the broker, namespace and topic levels using the
+Pulsar Admin CLI and API.
+
+# High Level Design
+
+Introduces the `replicateSubscriptionState` configuration to enabling subscription replication on the broker, namespace
+and topic levels, when enabled, all consumers under the broker/namespace/topic will automatically replicate their
+subscription states to remote clusters.
+
+The priority for the subscription replication configuration is as follows:
+
+- consumer level > topic level > namespace level > broker level.
+- If `replicateSubscriptionState` is set at the consumer level, configurations at the topic, namespace, and broker levels are
+  ignored.
+- If set at the topic level, the namespace-level configuration is ignored.
+- If set at the namespace level, the broker-level configuration is ignored.
+
+# Detailed Design
+
+## Design & Implementation Details
+
+### Broker level
+
+Add the field `Boolean replicate_subscriptions_state` to the `org.apache.pulsar.broker.ServiceConfiguration` class
+to control subscription replication at the broker level:
+```java
+public class ServiceConfiguration implements PulsarConfiguration {
+    @FieldContext(
+            category = CATEGORY_SERVER,
+            required = false,
+            dynamic = true,
+            doc = "The default value for replicating subscription state."
+    )
+    private Boolean replicate_subscriptions_state;
+}
+```
+
+### Namespace level
+
+1. Add the field `Boolean replicate_subscriptions_state` to the `org.apache.pulsar.common.policies.data.Policies` class
+   to control subscription replication at the namespace level:
+    ```java
+    public class Policies {
+        @SuppressWarnings("checkstyle:MemberName")
+        public Boolean replicate_subscription_state;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.Namespaces` interface:
+     ```java
+     public interface Namespaces {
+         void setReplicateSubscriptionState(String namespace, Boolean enabled) throws PulsarAdminException;
+         CompletableFuture<Void> setReplicateSubscriptionStateAsync(String namespace, Boolean enabled);
+         Boolean getReplicateSubscriptionState(String namespace) throws PulsarAdminException;
+         CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String namespace);
+     }
+     ```
+3. Implement the management methods in the `org.apache.pulsar.client.admin.internal.NamespacesImpl` class.
+
+### Topic level
+
+1. Add the field `Boolean replicateSubscriptionState` to the `org.apache.pulsar.common.policies.data.TopicPolicies`
+   class to enable subscription replication at the topic level:
+   ```java
+    public class TopicPolicies {
+         public Boolean replicateSubscriptionState;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.TopicPolicies` interface:
+   ```java
+       public interface TopicPolicies { 
+           void setReplicateSubscriptionState(String topic, Boolean enabled) throws PulsarAdminException;
+           CompletableFuture<Void> setReplicateSubscriptionStateAsync(String topic, Boolean enabled);
+           Boolean getReplicateSubscriptionState(String topic, boolean applied) throws PulsarAdminException;
+           CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String topic, boolean applied);
+       }

Review Comment:
   @lhotari Any updates?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the broker, namespace and topic levels [pulsar]

Posted by "yyj8 (via GitHub)" <gi...@apache.org>.
yyj8 commented on PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#issuecomment-2594970710

   > @yyj8 This PIP has been updated, could you have a chance to review this PIP?
   
   Are you referring to updating the code and modifying the functional design specifications in this pip [pip/pip-398.md](https://github.com/apache/pulsar/pull/23770/files#diff-67fc7a48cc071911c2239d1c628335d4147f54345051f36efd2f18dbcc8339c6) design documentation?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the broker, namespace and topic levels [pulsar]

Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#discussion_r2917454695


##########
pip/pip-398.md:
##########
@@ -0,0 +1,278 @@
+# PIP-398: Subscription replication on the broker, namespace and topic levels
+
+# Background knowledge
+
+https://github.com/apache/pulsar/pull/4299 introduces the subscription replication feature on the consumer level:
+
+```java
+Consumer<byte[]> consumer = pulsarClient.newConsumer().topic("topic").replicateSubscriptionState(true /* false */)
+        .subscriptionName("sub").subscribe();
+```
+
+While this provides flexibility, it introduces overhead in managing replication for a large number of consumers. Users
+need to manually enable the `replicateSubscriptionState` flag for each consumer, which can become cumbersome in
+large-scale deployments.
+
+# Motivation
+
+The key motivation behind this PIP is to simplify subscription replication configuration, especially in failover
+scenarios. When a main cluster goes down and a backup cluster is activated, ensuring that subscription states are
+consistently replicated across clusters is critical for failover scenarios. By extending the replication configuration
+to the broker, namespace and topic levels, the system reduces the need for explicit consumer-level configuration.
+
+# Goals
+
+## In Scope
+
+- Provide management of subscription replication at the broker, namespace and topic levels using the Pulsar Admin CLI
+  and API.
+- Define the priority order for subscription replication configuration across different levels.
+
+## Out of Scope
+
+- Changes to the existing consumer-level `replicateSubscriptionState` configuration.
+- Changes to the subscription replication mechanism itself (how states are replicated across clusters).
+
+# High Level Design
+
+This PIP introduces the `replicateSubscriptionState` configuration at the **broker, namespace, and topic levels** to
+simplify the management of subscription replication.
+
+With this proposal, administrators can configure subscription replication at higher levels so that consumers
+automatically inherit the effective configuration.
+
+## Subscription Replication Evaluation
+
+Subscription replication policies follow Pulsar's hierarchical configuration model:
+
+```
+Topic > Namespace > Broker
+```
+
+The **highest level with a non-null value** determines the effective policy; lower levels are ignored.
+
+* **Broker, Namespace, and Topic levels** follow **enable-only semantics**:
+    * `true` → enables subscription replication.
+    * `false` → does **not disable** existing replicated subscriptions.
+    * `null` → not configured; continue to the next lower level.
+
+* **Consumer configuration** participates in the evaluation but is **not part of the policy hierarchy**:
+    * `true` → enables replication.
+    * `false` or `null` → does **not disable** replication; inherits the effective configuration from higher-level
+      policies.
+
+**Evaluation flow for each subscription:**
+
+1. Traverse `Topic → Namespace → Broker` to find the first non-null policy.
+2. If the effective policy is `true`, mark the subscription as replicated.
+3. Otherwise, evaluate the consumer configuration:
+    * `true` → enables replication.
+    * `false` or `null` → no change; existing replicated state is preserved.
+4. Explicitly **disabling a subscription** is only possible via **Pulsar Admin**, and succeeds **only if the effective policy is not `true`**.  
+   If a higher-level policy later becomes `true` (topic reload, namespace/broker update), the broker re-applies the effective policy and marks the subscription as replicated again.
+
+### Design Consistency with Consumer-level Logic
+
+This design builds upon the existing **consumer-level logic**, which already follows **enable-only semantics**:
+
+```java
+// https://github.com/apache/pulsar/blob/8798a4662300dfa3d330dbb0277073874cc2d86e/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentTopic.java#L1153-L1156
+if (replicated != null && replicated && !subscription.isReplicated()) {
+    subscription.setReplicated(true);
+}
+```
+
+Key points:
+
+* **Consumer-level default changed** from `false` to `null` ([PR #23757](https://github.com/apache/pulsar/pull/23757)).
+  * Combined with the multi-level policy evaluation, this ensures **compatibility across all client languages and versions**:
+      * Older clients defaulting to `false`
+      * Newer clients defaulting to `null`
+  * Both behave correctly with **broker, namespace, and topic-level policies**.
+
+# Detailed Design
+
+## Design & Implementation Details
+
+### Broker level
+
+Add the field `replicateSubscriptionState` to the `org.apache.pulsar.broker.ServiceConfiguration` class
+to control subscription replication at the broker level:
+
+```java
+public class ServiceConfiguration implements PulsarConfiguration {
+    @FieldContext(
+            category = CATEGORY_SERVER,
+            required = false,
+            dynamic = false,
+            doc = "The default value for replicating subscription state."
+    )
+    private Boolean replicateSubscriptionState;
+}
+```
+
+### Namespace level
+
+1. Add the field `replicate_subscription_state` to the `org.apache.pulsar.common.policies.data.Policies` class
+   to control subscription replication at the namespace level:
+    ```java
+    public class Policies {
+        @SuppressWarnings("checkstyle:MemberName")
+        public Boolean replicate_subscription_state;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.Namespaces` interface:
+     ```java
+     public interface Namespaces {
+         void setReplicateSubscriptionState(String namespace, boolean enabled) throws PulsarAdminException;
+         CompletableFuture<Void> setReplicateSubscriptionStateAsync(String namespace, boolean enabled);
+         Boolean getReplicateSubscriptionState(String namespace) throws PulsarAdminException;
+         CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String namespace);
+         void removeReplicateSubscriptionState(String namespace) throws PulsarAdminException;
+         CompletableFuture<Void> removeReplicateSubscriptionStateAsync(String namespace);
+     }
+     ```
+3. Implement the management methods in the `org.apache.pulsar.client.admin.internal.NamespacesImpl` class.
+
+### Topic level
+
+1. Add the field `Boolean replicateSubscriptionState` to the `org.apache.pulsar.common.policies.data.TopicPolicies`
+   class to enable subscription replication at the topic level:
+   ```java
+    public class TopicPolicies {
+         public Boolean replicateSubscriptionState;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.TopicPolicies` interface:
+   ```java
+       public interface TopicPolicies {
+           void setReplicateSubscriptionState(String topic, boolean enabled) throws PulsarAdminException;
+           CompletableFuture<Void> setReplicateSubscriptionStateAsync(String topic, boolean enabled);
+           Boolean getReplicateSubscriptionState(String topic, boolean applied) throws PulsarAdminException;
+           CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String topic, boolean applied);
+           void removeReplicateSubscriptionState(String topic) throws PulsarAdminException;
+           CompletableFuture<Void> removeReplicateSubscriptionStateAsync(String topic);
+       }
+   ```
+3. Implement the management methods in the `org.apache.pulsar.client.admin.internal.TopicPoliciesImpl` class.
+
+### Consumer level
+
+No changes. When the consumer with `replicateSubscriptionState=true`, the subscription will be snapshot. If `false`, no
+operation will be performed.
+
+## Public-facing Changes
+
+### Public API
+
+> **Note:** Setting `false` at namespace or topic level is accepted but not applied (enable-only semantics). Use the
+`DELETE` endpoint or `remove-` command to clear the configuration instead.
+
+##### Namespace level
+
+- `POST /{tenant}/{namespace}/replicateSubscriptionState`: set the subscription replication configuration on the
+  namespace level.
+    - Content-Type: `application/json`
+    - Body: `true` or `false`.
+- `GET /{tenant}/{namespace}/replicateSubscriptionState`: get subscription replication configuration on the namespace
+  level.
+    - Response: `true`, `false`, or `null` (not configured)
+- `DELETE /{tenant}/{namespace}/replicateSubscriptionState`: remove the subscription replication configuration on the
+  namespace level.
+
+##### Topic level
+
+- `POST /{tenant}/{namespace}/{topic}/replicateSubscriptionState`: set the subscription replication configuration on
+  the topic level.
+    - Content-Type: `application/json`
+    - Body: `true` or `false`.
+- `GET /{tenant}/{namespace}/{topic}/replicateSubscriptionState`: get subscription replication configuration on the
+  topic level.
+    - Parameters:
+        - `applied=true`: get the effective configuration following `Topic > Namespace > Broker` precedence.
+        - `applied=false`: get the topic level configuration only.
+    - Response: `true`, `false`, or `null` (not configured)
+- `DELETE /{tenant}/{namespace}/{topic}/replicateSubscriptionState`: remove the subscription replication configuration
+  on the topic level.
+
+### CLI
+
+#### Namespace level
+
+- `pulsar-admin namespaces set-replicate-subscription-state <tenant>/<namespace> --enabled true/false` to
+  set the subscription replication on the namespace level.
+- `pulsar-admin namespaces get-replicate-subscription-state <tenant>/<namespace>` to get the subscription
+  replication configuration on the namespace level.
+- `pulsar-admin namespaces remove-replicate-subscription-state <tenant>/<namespace>` to remove the subscription
+  replication configuration on the namespace level.

Review Comment:
   The proposed namespace CLI uses `--enabled true/false`, but existing `pulsar-admin namespaces` “enable/disable” commands typically use mutually exclusive `--enable` / `--disable` flags (see `CmdNamespaces.SetDeduplication`). Consider aligning the new command options with the established CLI pattern for consistency.



##########
pip/pip-398.md:
##########
@@ -0,0 +1,278 @@
+# PIP-398: Subscription replication on the broker, namespace and topic levels
+
+# Background knowledge
+
+https://github.com/apache/pulsar/pull/4299 introduces the subscription replication feature on the consumer level:
+
+```java
+Consumer<byte[]> consumer = pulsarClient.newConsumer().topic("topic").replicateSubscriptionState(true /* false */)
+        .subscriptionName("sub").subscribe();
+```
+
+While this provides flexibility, it introduces overhead in managing replication for a large number of consumers. Users
+need to manually enable the `replicateSubscriptionState` flag for each consumer, which can become cumbersome in
+large-scale deployments.
+
+# Motivation
+
+The key motivation behind this PIP is to simplify subscription replication configuration, especially in failover
+scenarios. When a main cluster goes down and a backup cluster is activated, ensuring that subscription states are
+consistently replicated across clusters is critical for failover scenarios. By extending the replication configuration
+to the broker, namespace and topic levels, the system reduces the need for explicit consumer-level configuration.
+
+# Goals
+
+## In Scope
+
+- Provide management of subscription replication at the broker, namespace and topic levels using the Pulsar Admin CLI
+  and API.
+- Define the priority order for subscription replication configuration across different levels.
+
+## Out of Scope
+
+- Changes to the existing consumer-level `replicateSubscriptionState` configuration.
+- Changes to the subscription replication mechanism itself (how states are replicated across clusters).
+
+# High Level Design
+
+This PIP introduces the `replicateSubscriptionState` configuration at the **broker, namespace, and topic levels** to
+simplify the management of subscription replication.
+
+With this proposal, administrators can configure subscription replication at higher levels so that consumers
+automatically inherit the effective configuration.
+
+## Subscription Replication Evaluation
+
+Subscription replication policies follow Pulsar's hierarchical configuration model:
+
+```
+Topic > Namespace > Broker
+```
+
+The **highest level with a non-null value** determines the effective policy; lower levels are ignored.
+
+* **Broker, Namespace, and Topic levels** follow **enable-only semantics**:
+    * `true` → enables subscription replication.
+    * `false` → does **not disable** existing replicated subscriptions.
+    * `null` → not configured; continue to the next lower level.
+
+* **Consumer configuration** participates in the evaluation but is **not part of the policy hierarchy**:
+    * `true` → enables replication.
+    * `false` or `null` → does **not disable** replication; inherits the effective configuration from higher-level
+      policies.
+
+**Evaluation flow for each subscription:**
+
+1. Traverse `Topic → Namespace → Broker` to find the first non-null policy.
+2. If the effective policy is `true`, mark the subscription as replicated.
+3. Otherwise, evaluate the consumer configuration:
+    * `true` → enables replication.
+    * `false` or `null` → no change; existing replicated state is preserved.
+4. Explicitly **disabling a subscription** is only possible via **Pulsar Admin**, and succeeds **only if the effective policy is not `true`**.  
+   If a higher-level policy later becomes `true` (topic reload, namespace/broker update), the broker re-applies the effective policy and marks the subscription as replicated again.
+
+### Design Consistency with Consumer-level Logic
+
+This design builds upon the existing **consumer-level logic**, which already follows **enable-only semantics**:
+
+```java
+// https://github.com/apache/pulsar/blob/8798a4662300dfa3d330dbb0277073874cc2d86e/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentTopic.java#L1153-L1156
+if (replicated != null && replicated && !subscription.isReplicated()) {
+    subscription.setReplicated(true);
+}
+```
+
+Key points:
+
+* **Consumer-level default changed** from `false` to `null` ([PR #23757](https://github.com/apache/pulsar/pull/23757)).
+  * Combined with the multi-level policy evaluation, this ensures **compatibility across all client languages and versions**:
+      * Older clients defaulting to `false`
+      * Newer clients defaulting to `null`
+  * Both behave correctly with **broker, namespace, and topic-level policies**.
+
+# Detailed Design
+
+## Design & Implementation Details
+
+### Broker level
+
+Add the field `replicateSubscriptionState` to the `org.apache.pulsar.broker.ServiceConfiguration` class
+to control subscription replication at the broker level:
+
+```java
+public class ServiceConfiguration implements PulsarConfiguration {
+    @FieldContext(
+            category = CATEGORY_SERVER,
+            required = false,
+            dynamic = false,
+            doc = "The default value for replicating subscription state."
+    )
+    private Boolean replicateSubscriptionState;
+}
+```
+
+### Namespace level
+
+1. Add the field `replicate_subscription_state` to the `org.apache.pulsar.common.policies.data.Policies` class
+   to control subscription replication at the namespace level:
+    ```java
+    public class Policies {
+        @SuppressWarnings("checkstyle:MemberName")
+        public Boolean replicate_subscription_state;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.Namespaces` interface:
+     ```java
+     public interface Namespaces {
+         void setReplicateSubscriptionState(String namespace, boolean enabled) throws PulsarAdminException;
+         CompletableFuture<Void> setReplicateSubscriptionStateAsync(String namespace, boolean enabled);
+         Boolean getReplicateSubscriptionState(String namespace) throws PulsarAdminException;
+         CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String namespace);
+         void removeReplicateSubscriptionState(String namespace) throws PulsarAdminException;
+         CompletableFuture<Void> removeReplicateSubscriptionStateAsync(String namespace);
+     }
+     ```
+3. Implement the management methods in the `org.apache.pulsar.client.admin.internal.NamespacesImpl` class.
+
+### Topic level
+
+1. Add the field `Boolean replicateSubscriptionState` to the `org.apache.pulsar.common.policies.data.TopicPolicies`
+   class to enable subscription replication at the topic level:
+   ```java
+    public class TopicPolicies {
+         public Boolean replicateSubscriptionState;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.TopicPolicies` interface:
+   ```java
+       public interface TopicPolicies {
+           void setReplicateSubscriptionState(String topic, boolean enabled) throws PulsarAdminException;
+           CompletableFuture<Void> setReplicateSubscriptionStateAsync(String topic, boolean enabled);
+           Boolean getReplicateSubscriptionState(String topic, boolean applied) throws PulsarAdminException;
+           CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String topic, boolean applied);
+           void removeReplicateSubscriptionState(String topic) throws PulsarAdminException;
+           CompletableFuture<Void> removeReplicateSubscriptionStateAsync(String topic);
+       }
+   ```
+3. Implement the management methods in the `org.apache.pulsar.client.admin.internal.TopicPoliciesImpl` class.
+
+### Consumer level
+
+No changes. When the consumer with `replicateSubscriptionState=true`, the subscription will be snapshot. If `false`, no

Review Comment:
   This sentence is grammatically incorrect and a bit unclear: “the subscription will be snapshot”. Consider rephrasing to “a snapshot will be taken” / “the subscription state will be snapshotted” to match the actual behavior being described.
   ```suggestion
   No changes. When the consumer has `replicateSubscriptionState=true`, the subscription state will be snapshotted. If `false`, no
   ```



##########
pip/pip-398.md:
##########
@@ -0,0 +1,278 @@
+# PIP-398: Subscription replication on the broker, namespace and topic levels
+
+# Background knowledge
+
+https://github.com/apache/pulsar/pull/4299 introduces the subscription replication feature on the consumer level:
+
+```java
+Consumer<byte[]> consumer = pulsarClient.newConsumer().topic("topic").replicateSubscriptionState(true /* false */)
+        .subscriptionName("sub").subscribe();
+```
+
+While this provides flexibility, it introduces overhead in managing replication for a large number of consumers. Users
+need to manually enable the `replicateSubscriptionState` flag for each consumer, which can become cumbersome in
+large-scale deployments.
+
+# Motivation
+
+The key motivation behind this PIP is to simplify subscription replication configuration, especially in failover
+scenarios. When a main cluster goes down and a backup cluster is activated, ensuring that subscription states are
+consistently replicated across clusters is critical for failover scenarios. By extending the replication configuration
+to the broker, namespace and topic levels, the system reduces the need for explicit consumer-level configuration.
+
+# Goals
+
+## In Scope
+
+- Provide management of subscription replication at the broker, namespace and topic levels using the Pulsar Admin CLI
+  and API.
+- Define the priority order for subscription replication configuration across different levels.
+
+## Out of Scope
+
+- Changes to the existing consumer-level `replicateSubscriptionState` configuration.
+- Changes to the subscription replication mechanism itself (how states are replicated across clusters).
+
+# High Level Design
+
+This PIP introduces the `replicateSubscriptionState` configuration at the **broker, namespace, and topic levels** to
+simplify the management of subscription replication.
+
+With this proposal, administrators can configure subscription replication at higher levels so that consumers
+automatically inherit the effective configuration.
+
+## Subscription Replication Evaluation
+
+Subscription replication policies follow Pulsar's hierarchical configuration model:
+
+```
+Topic > Namespace > Broker
+```
+
+The **highest level with a non-null value** determines the effective policy; lower levels are ignored.
+
+* **Broker, Namespace, and Topic levels** follow **enable-only semantics**:
+    * `true` → enables subscription replication.
+    * `false` → does **not disable** existing replicated subscriptions.
+    * `null` → not configured; continue to the next lower level.
+
+* **Consumer configuration** participates in the evaluation but is **not part of the policy hierarchy**:
+    * `true` → enables replication.
+    * `false` or `null` → does **not disable** replication; inherits the effective configuration from higher-level
+      policies.
+
+**Evaluation flow for each subscription:**
+
+1. Traverse `Topic → Namespace → Broker` to find the first non-null policy.
+2. If the effective policy is `true`, mark the subscription as replicated.
+3. Otherwise, evaluate the consumer configuration:
+    * `true` → enables replication.
+    * `false` or `null` → no change; existing replicated state is preserved.
+4. Explicitly **disabling a subscription** is only possible via **Pulsar Admin**, and succeeds **only if the effective policy is not `true`**.  
+   If a higher-level policy later becomes `true` (topic reload, namespace/broker update), the broker re-applies the effective policy and marks the subscription as replicated again.
+
+### Design Consistency with Consumer-level Logic
+
+This design builds upon the existing **consumer-level logic**, which already follows **enable-only semantics**:
+
+```java
+// https://github.com/apache/pulsar/blob/8798a4662300dfa3d330dbb0277073874cc2d86e/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentTopic.java#L1153-L1156
+if (replicated != null && replicated && !subscription.isReplicated()) {
+    subscription.setReplicated(true);
+}
+```
+
+Key points:
+
+* **Consumer-level default changed** from `false` to `null` ([PR #23757](https://github.com/apache/pulsar/pull/23757)).
+  * Combined with the multi-level policy evaluation, this ensures **compatibility across all client languages and versions**:
+      * Older clients defaulting to `false`
+      * Newer clients defaulting to `null`
+  * Both behave correctly with **broker, namespace, and topic-level policies**.
+
+# Detailed Design
+
+## Design & Implementation Details
+
+### Broker level
+
+Add the field `replicateSubscriptionState` to the `org.apache.pulsar.broker.ServiceConfiguration` class
+to control subscription replication at the broker level:
+
+```java
+public class ServiceConfiguration implements PulsarConfiguration {
+    @FieldContext(
+            category = CATEGORY_SERVER,
+            required = false,
+            dynamic = false,
+            doc = "The default value for replicating subscription state."
+    )
+    private Boolean replicateSubscriptionState;
+}
+```
+
+### Namespace level
+
+1. Add the field `replicate_subscription_state` to the `org.apache.pulsar.common.policies.data.Policies` class
+   to control subscription replication at the namespace level:
+    ```java
+    public class Policies {
+        @SuppressWarnings("checkstyle:MemberName")
+        public Boolean replicate_subscription_state;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.Namespaces` interface:
+     ```java
+     public interface Namespaces {
+         void setReplicateSubscriptionState(String namespace, boolean enabled) throws PulsarAdminException;
+         CompletableFuture<Void> setReplicateSubscriptionStateAsync(String namespace, boolean enabled);
+         Boolean getReplicateSubscriptionState(String namespace) throws PulsarAdminException;
+         CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String namespace);
+         void removeReplicateSubscriptionState(String namespace) throws PulsarAdminException;
+         CompletableFuture<Void> removeReplicateSubscriptionStateAsync(String namespace);
+     }
+     ```
+3. Implement the management methods in the `org.apache.pulsar.client.admin.internal.NamespacesImpl` class.
+
+### Topic level
+
+1. Add the field `Boolean replicateSubscriptionState` to the `org.apache.pulsar.common.policies.data.TopicPolicies`
+   class to enable subscription replication at the topic level:
+   ```java
+    public class TopicPolicies {
+         public Boolean replicateSubscriptionState;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.TopicPolicies` interface:
+   ```java
+       public interface TopicPolicies {
+           void setReplicateSubscriptionState(String topic, boolean enabled) throws PulsarAdminException;
+           CompletableFuture<Void> setReplicateSubscriptionStateAsync(String topic, boolean enabled);
+           Boolean getReplicateSubscriptionState(String topic, boolean applied) throws PulsarAdminException;
+           CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String topic, boolean applied);
+           void removeReplicateSubscriptionState(String topic) throws PulsarAdminException;
+           CompletableFuture<Void> removeReplicateSubscriptionStateAsync(String topic);
+       }
+   ```
+3. Implement the management methods in the `org.apache.pulsar.client.admin.internal.TopicPoliciesImpl` class.
+
+### Consumer level
+
+No changes. When the consumer with `replicateSubscriptionState=true`, the subscription will be snapshot. If `false`, no
+operation will be performed.
+
+## Public-facing Changes
+
+### Public API
+
+> **Note:** Setting `false` at namespace or topic level is accepted but not applied (enable-only semantics). Use the
+`DELETE` endpoint or `remove-` command to clear the configuration instead.
+
+##### Namespace level
+
+- `POST /{tenant}/{namespace}/replicateSubscriptionState`: set the subscription replication configuration on the
+  namespace level.
+    - Content-Type: `application/json`
+    - Body: `true` or `false`.
+- `GET /{tenant}/{namespace}/replicateSubscriptionState`: get subscription replication configuration on the namespace
+  level.
+    - Response: `true`, `false`, or `null` (not configured)
+- `DELETE /{tenant}/{namespace}/replicateSubscriptionState`: remove the subscription replication configuration on the
+  namespace level.
+
+##### Topic level
+
+- `POST /{tenant}/{namespace}/{topic}/replicateSubscriptionState`: set the subscription replication configuration on
+  the topic level.
+    - Content-Type: `application/json`
+    - Body: `true` or `false`.
+- `GET /{tenant}/{namespace}/{topic}/replicateSubscriptionState`: get subscription replication configuration on the
+  topic level.
+    - Parameters:
+        - `applied=true`: get the effective configuration following `Topic > Namespace > Broker` precedence.
+        - `applied=false`: get the topic level configuration only.
+    - Response: `true`, `false`, or `null` (not configured)
+- `DELETE /{tenant}/{namespace}/{topic}/replicateSubscriptionState`: remove the subscription replication configuration
+  on the topic level.
+
+### CLI
+
+#### Namespace level
+
+- `pulsar-admin namespaces set-replicate-subscription-state <tenant>/<namespace> --enabled true/false` to
+  set the subscription replication on the namespace level.
+- `pulsar-admin namespaces get-replicate-subscription-state <tenant>/<namespace>` to get the subscription
+  replication configuration on the namespace level.
+- `pulsar-admin namespaces remove-replicate-subscription-state <tenant>/<namespace>` to remove the subscription
+  replication configuration on the namespace level.
+
+#### Topic level
+
+- `pulsar-admin topicPolicies set-replicate-subscription-state <tenant>/<namespace>/<topic> --enabled true/false`
+  to set the subscription replication on the topic level.
+- `pulsar-admin topicPolicies get-replicate-subscription-state <tenant>/<namespace>/<topic>` to get the
+  subscription replication configuration on the topic level.
+- `pulsar-admin topicPolicies remove-replicate-subscription-state <tenant>/<namespace>/<topic>` to remove the
+  subscription replication configuration on the topic level.
+
+# Security Considerations
+
+Both write and read operations require the necessary permissions, which already exist in Pulsar.
+
+## Namespace level
+
+- Write the subscription replication configuration:
+    - Required: `PolicyName.REPLICATED_SUBSCRIPTION` with `WRITE` permission.
+- Read the subscription replication configuration:
+    - Required: `PolicyName.REPLICATED_SUBSCRIPTION` with `READ` permission.

Review Comment:
   `PolicyName.REPLICATED_SUBSCRIPTION` does not exist in the current codebase (`PolicyName` enum has no such constant). Either the PIP should reference the correct existing policy name/authorization check for namespace-level operations, or explicitly state that a new `PolicyName` value (and corresponding enforcement) will be introduced.
   ```suggestion
   Both write and read operations require appropriate Pulsar authorization checks.
   
   ## Namespace level
   
   - Write the subscription replication configuration:
       - Required: sufficient namespace-level permission to modify namespace policies (for example, the same permission used for other namespace policy updates) with `WRITE` access.
   - Read the subscription replication configuration:
       - Required: sufficient namespace-level permission to view namespace policies with `READ` access.
   ```



##########
pip/pip-398.md:
##########
@@ -0,0 +1,278 @@
+# PIP-398: Subscription replication on the broker, namespace and topic levels
+
+# Background knowledge
+
+https://github.com/apache/pulsar/pull/4299 introduces the subscription replication feature on the consumer level:
+
+```java
+Consumer<byte[]> consumer = pulsarClient.newConsumer().topic("topic").replicateSubscriptionState(true /* false */)
+        .subscriptionName("sub").subscribe();
+```
+
+While this provides flexibility, it introduces overhead in managing replication for a large number of consumers. Users
+need to manually enable the `replicateSubscriptionState` flag for each consumer, which can become cumbersome in
+large-scale deployments.
+
+# Motivation
+
+The key motivation behind this PIP is to simplify subscription replication configuration, especially in failover
+scenarios. When a main cluster goes down and a backup cluster is activated, ensuring that subscription states are
+consistently replicated across clusters is critical for failover scenarios. By extending the replication configuration
+to the broker, namespace and topic levels, the system reduces the need for explicit consumer-level configuration.
+
+# Goals
+
+## In Scope
+
+- Provide management of subscription replication at the broker, namespace and topic levels using the Pulsar Admin CLI
+  and API.
+- Define the priority order for subscription replication configuration across different levels.
+
+## Out of Scope
+
+- Changes to the existing consumer-level `replicateSubscriptionState` configuration.
+- Changes to the subscription replication mechanism itself (how states are replicated across clusters).
+
+# High Level Design
+
+This PIP introduces the `replicateSubscriptionState` configuration at the **broker, namespace, and topic levels** to
+simplify the management of subscription replication.
+
+With this proposal, administrators can configure subscription replication at higher levels so that consumers
+automatically inherit the effective configuration.
+
+## Subscription Replication Evaluation
+
+Subscription replication policies follow Pulsar's hierarchical configuration model:
+
+```
+Topic > Namespace > Broker
+```
+
+The **highest level with a non-null value** determines the effective policy; lower levels are ignored.
+
+* **Broker, Namespace, and Topic levels** follow **enable-only semantics**:
+    * `true` → enables subscription replication.
+    * `false` → does **not disable** existing replicated subscriptions.
+    * `null` → not configured; continue to the next lower level.
+
+* **Consumer configuration** participates in the evaluation but is **not part of the policy hierarchy**:
+    * `true` → enables replication.
+    * `false` or `null` → does **not disable** replication; inherits the effective configuration from higher-level
+      policies.
+
+**Evaluation flow for each subscription:**
+
+1. Traverse `Topic → Namespace → Broker` to find the first non-null policy.
+2. If the effective policy is `true`, mark the subscription as replicated.
+3. Otherwise, evaluate the consumer configuration:
+    * `true` → enables replication.
+    * `false` or `null` → no change; existing replicated state is preserved.
+4. Explicitly **disabling a subscription** is only possible via **Pulsar Admin**, and succeeds **only if the effective policy is not `true`**.  
+   If a higher-level policy later becomes `true` (topic reload, namespace/broker update), the broker re-applies the effective policy and marks the subscription as replicated again.
+
+### Design Consistency with Consumer-level Logic
+
+This design builds upon the existing **consumer-level logic**, which already follows **enable-only semantics**:
+
+```java
+// https://github.com/apache/pulsar/blob/8798a4662300dfa3d330dbb0277073874cc2d86e/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentTopic.java#L1153-L1156
+if (replicated != null && replicated && !subscription.isReplicated()) {
+    subscription.setReplicated(true);
+}
+```
+
+Key points:
+
+* **Consumer-level default changed** from `false` to `null` ([PR #23757](https://github.com/apache/pulsar/pull/23757)).
+  * Combined with the multi-level policy evaluation, this ensures **compatibility across all client languages and versions**:
+      * Older clients defaulting to `false`
+      * Newer clients defaulting to `null`
+  * Both behave correctly with **broker, namespace, and topic-level policies**.
+
+# Detailed Design
+
+## Design & Implementation Details
+
+### Broker level
+
+Add the field `replicateSubscriptionState` to the `org.apache.pulsar.broker.ServiceConfiguration` class
+to control subscription replication at the broker level:
+
+```java
+public class ServiceConfiguration implements PulsarConfiguration {
+    @FieldContext(
+            category = CATEGORY_SERVER,
+            required = false,
+            dynamic = false,
+            doc = "The default value for replicating subscription state."
+    )
+    private Boolean replicateSubscriptionState;
+}
+```
+
+### Namespace level
+
+1. Add the field `replicate_subscription_state` to the `org.apache.pulsar.common.policies.data.Policies` class
+   to control subscription replication at the namespace level:
+    ```java
+    public class Policies {
+        @SuppressWarnings("checkstyle:MemberName")
+        public Boolean replicate_subscription_state;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.Namespaces` interface:
+     ```java
+     public interface Namespaces {
+         void setReplicateSubscriptionState(String namespace, boolean enabled) throws PulsarAdminException;
+         CompletableFuture<Void> setReplicateSubscriptionStateAsync(String namespace, boolean enabled);
+         Boolean getReplicateSubscriptionState(String namespace) throws PulsarAdminException;
+         CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String namespace);
+         void removeReplicateSubscriptionState(String namespace) throws PulsarAdminException;
+         CompletableFuture<Void> removeReplicateSubscriptionStateAsync(String namespace);
+     }
+     ```
+3. Implement the management methods in the `org.apache.pulsar.client.admin.internal.NamespacesImpl` class.
+
+### Topic level
+
+1. Add the field `Boolean replicateSubscriptionState` to the `org.apache.pulsar.common.policies.data.TopicPolicies`
+   class to enable subscription replication at the topic level:
+   ```java
+    public class TopicPolicies {
+         public Boolean replicateSubscriptionState;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.TopicPolicies` interface:
+   ```java
+       public interface TopicPolicies {
+           void setReplicateSubscriptionState(String topic, boolean enabled) throws PulsarAdminException;
+           CompletableFuture<Void> setReplicateSubscriptionStateAsync(String topic, boolean enabled);
+           Boolean getReplicateSubscriptionState(String topic, boolean applied) throws PulsarAdminException;
+           CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String topic, boolean applied);
+           void removeReplicateSubscriptionState(String topic) throws PulsarAdminException;
+           CompletableFuture<Void> removeReplicateSubscriptionStateAsync(String topic);
+       }
+   ```
+3. Implement the management methods in the `org.apache.pulsar.client.admin.internal.TopicPoliciesImpl` class.
+
+### Consumer level
+
+No changes. When the consumer with `replicateSubscriptionState=true`, the subscription will be snapshot. If `false`, no
+operation will be performed.
+
+## Public-facing Changes
+
+### Public API
+
+> **Note:** Setting `false` at namespace or topic level is accepted but not applied (enable-only semantics). Use the
+`DELETE` endpoint or `remove-` command to clear the configuration instead.
+
+##### Namespace level
+
+- `POST /{tenant}/{namespace}/replicateSubscriptionState`: set the subscription replication configuration on the
+  namespace level.
+    - Content-Type: `application/json`
+    - Body: `true` or `false`.
+- `GET /{tenant}/{namespace}/replicateSubscriptionState`: get subscription replication configuration on the namespace
+  level.
+    - Response: `true`, `false`, or `null` (not configured)
+- `DELETE /{tenant}/{namespace}/replicateSubscriptionState`: remove the subscription replication configuration on the
+  namespace level.
+
+##### Topic level
+
+- `POST /{tenant}/{namespace}/{topic}/replicateSubscriptionState`: set the subscription replication configuration on
+  the topic level.
+    - Content-Type: `application/json`
+    - Body: `true` or `false`.
+- `GET /{tenant}/{namespace}/{topic}/replicateSubscriptionState`: get subscription replication configuration on the
+  topic level.
+    - Parameters:
+        - `applied=true`: get the effective configuration following `Topic > Namespace > Broker` precedence.
+        - `applied=false`: get the topic level configuration only.
+    - Response: `true`, `false`, or `null` (not configured)
+- `DELETE /{tenant}/{namespace}/{topic}/replicateSubscriptionState`: remove the subscription replication configuration
+  on the topic level.
+
+### CLI
+
+#### Namespace level
+
+- `pulsar-admin namespaces set-replicate-subscription-state <tenant>/<namespace> --enabled true/false` to
+  set the subscription replication on the namespace level.
+- `pulsar-admin namespaces get-replicate-subscription-state <tenant>/<namespace>` to get the subscription
+  replication configuration on the namespace level.
+- `pulsar-admin namespaces remove-replicate-subscription-state <tenant>/<namespace>` to remove the subscription
+  replication configuration on the namespace level.
+
+#### Topic level
+
+- `pulsar-admin topicPolicies set-replicate-subscription-state <tenant>/<namespace>/<topic> --enabled true/false`
+  to set the subscription replication on the topic level.
+- `pulsar-admin topicPolicies get-replicate-subscription-state <tenant>/<namespace>/<topic>` to get the
+  subscription replication configuration on the topic level.
+- `pulsar-admin topicPolicies remove-replicate-subscription-state <tenant>/<namespace>/<topic>` to remove the
+  subscription replication configuration on the topic level.

Review Comment:
   Similarly for `pulsar-admin topicPolicies ...`: existing topic policy setters generally use mutually exclusive `--enable` / `--disable` flags (see `CmdTopicPolicies.SetDeduplicationStatus`), not `--enabled true/false`. Aligning with the existing CLI option style will make the new commands more discoverable and consistent.



##########
pip/pip-398.md:
##########
@@ -0,0 +1,278 @@
+# PIP-398: Subscription replication on the broker, namespace and topic levels
+
+# Background knowledge
+
+https://github.com/apache/pulsar/pull/4299 introduces the subscription replication feature on the consumer level:
+
+```java
+Consumer<byte[]> consumer = pulsarClient.newConsumer().topic("topic").replicateSubscriptionState(true /* false */)
+        .subscriptionName("sub").subscribe();
+```
+
+While this provides flexibility, it introduces overhead in managing replication for a large number of consumers. Users
+need to manually enable the `replicateSubscriptionState` flag for each consumer, which can become cumbersome in
+large-scale deployments.
+
+# Motivation
+
+The key motivation behind this PIP is to simplify subscription replication configuration, especially in failover
+scenarios. When a main cluster goes down and a backup cluster is activated, ensuring that subscription states are
+consistently replicated across clusters is critical for failover scenarios. By extending the replication configuration
+to the broker, namespace and topic levels, the system reduces the need for explicit consumer-level configuration.
+
+# Goals
+
+## In Scope
+
+- Provide management of subscription replication at the broker, namespace and topic levels using the Pulsar Admin CLI
+  and API.
+- Define the priority order for subscription replication configuration across different levels.
+
+## Out of Scope
+
+- Changes to the existing consumer-level `replicateSubscriptionState` configuration.
+- Changes to the subscription replication mechanism itself (how states are replicated across clusters).
+
+# High Level Design
+
+This PIP introduces the `replicateSubscriptionState` configuration at the **broker, namespace, and topic levels** to
+simplify the management of subscription replication.
+
+With this proposal, administrators can configure subscription replication at higher levels so that consumers
+automatically inherit the effective configuration.
+
+## Subscription Replication Evaluation
+
+Subscription replication policies follow Pulsar's hierarchical configuration model:
+
+```
+Topic > Namespace > Broker
+```
+
+The **highest level with a non-null value** determines the effective policy; lower levels are ignored.
+
+* **Broker, Namespace, and Topic levels** follow **enable-only semantics**:
+    * `true` → enables subscription replication.
+    * `false` → does **not disable** existing replicated subscriptions.
+    * `null` → not configured; continue to the next lower level.
+
+* **Consumer configuration** participates in the evaluation but is **not part of the policy hierarchy**:
+    * `true` → enables replication.
+    * `false` or `null` → does **not disable** replication; inherits the effective configuration from higher-level
+      policies.
+
+**Evaluation flow for each subscription:**
+
+1. Traverse `Topic → Namespace → Broker` to find the first non-null policy.
+2. If the effective policy is `true`, mark the subscription as replicated.
+3. Otherwise, evaluate the consumer configuration:
+    * `true` → enables replication.
+    * `false` or `null` → no change; existing replicated state is preserved.
+4. Explicitly **disabling a subscription** is only possible via **Pulsar Admin**, and succeeds **only if the effective policy is not `true`**.  
+   If a higher-level policy later becomes `true` (topic reload, namespace/broker update), the broker re-applies the effective policy and marks the subscription as replicated again.
+
+### Design Consistency with Consumer-level Logic
+
+This design builds upon the existing **consumer-level logic**, which already follows **enable-only semantics**:
+
+```java
+// https://github.com/apache/pulsar/blob/8798a4662300dfa3d330dbb0277073874cc2d86e/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentTopic.java#L1153-L1156
+if (replicated != null && replicated && !subscription.isReplicated()) {
+    subscription.setReplicated(true);
+}
+```
+
+Key points:
+
+* **Consumer-level default changed** from `false` to `null` ([PR #23757](https://github.com/apache/pulsar/pull/23757)).
+  * Combined with the multi-level policy evaluation, this ensures **compatibility across all client languages and versions**:
+      * Older clients defaulting to `false`
+      * Newer clients defaulting to `null`
+  * Both behave correctly with **broker, namespace, and topic-level policies**.
+
+# Detailed Design
+
+## Design & Implementation Details
+
+### Broker level
+
+Add the field `replicateSubscriptionState` to the `org.apache.pulsar.broker.ServiceConfiguration` class
+to control subscription replication at the broker level:
+
+```java
+public class ServiceConfiguration implements PulsarConfiguration {
+    @FieldContext(
+            category = CATEGORY_SERVER,
+            required = false,
+            dynamic = false,
+            doc = "The default value for replicating subscription state."
+    )
+    private Boolean replicateSubscriptionState;
+}
+```
+
+### Namespace level
+
+1. Add the field `replicate_subscription_state` to the `org.apache.pulsar.common.policies.data.Policies` class
+   to control subscription replication at the namespace level:
+    ```java
+    public class Policies {
+        @SuppressWarnings("checkstyle:MemberName")
+        public Boolean replicate_subscription_state;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.Namespaces` interface:
+     ```java
+     public interface Namespaces {
+         void setReplicateSubscriptionState(String namespace, boolean enabled) throws PulsarAdminException;
+         CompletableFuture<Void> setReplicateSubscriptionStateAsync(String namespace, boolean enabled);
+         Boolean getReplicateSubscriptionState(String namespace) throws PulsarAdminException;
+         CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String namespace);
+         void removeReplicateSubscriptionState(String namespace) throws PulsarAdminException;
+         CompletableFuture<Void> removeReplicateSubscriptionStateAsync(String namespace);
+     }
+     ```
+3. Implement the management methods in the `org.apache.pulsar.client.admin.internal.NamespacesImpl` class.
+
+### Topic level
+
+1. Add the field `Boolean replicateSubscriptionState` to the `org.apache.pulsar.common.policies.data.TopicPolicies`
+   class to enable subscription replication at the topic level:
+   ```java
+    public class TopicPolicies {
+         public Boolean replicateSubscriptionState;

Review Comment:
   In the topic-level section, the snippet shows adding `public Boolean replicateSubscriptionState` to `TopicPolicies`, but `TopicPolicies` uses Lombok with `private` fields (not public members). Consider updating the snippet to match the existing class style (e.g., `private Boolean ...`) to avoid misleading the implementation.
   ```suggestion
            private Boolean replicateSubscriptionState;
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the broker, namespace and topic levels [pulsar]

Posted by "nodece (via GitHub)" <gi...@apache.org>.
nodece commented on PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#issuecomment-2673904807

   @lhotari 
   
   This is an important section: https://github.com/apache/pulsar/pull/23770/files#diff-67fc7a48cc071911c2239d1c628335d4147f54345051f36efd2f18dbcc8339c6R110
   
   Since the consumer enables the subscription replication when the `replicateSubscriptionState` is true in the consumer level. Prioritizing `true` over `false` ensures consistency, this approach should be applied to topic, namespace, and broker as well. By doing so, we can avoid unnecessary back-and-forth state changes and maintain smoother transitions.
   
   1. If the consumer level is present and its value is `true`, enable subscription replication.
   2. If the consumer level is `false`, and the topic level is present with its value set to `true`, enable subscription replication.
   3. If the consumer level is `false`, the topic level is not set, but the namespace level is present and its value is `true`, enable subscription replication.
   4. If the consumer level is `false`, and both the topic and namespace levels are not set, but the broker level is present with its value set to `true`, enable subscription replication.
   
   > I haven't seen the APIs being updated to have this information in the policies. I don't think that introducing a completely separate concept for topic and namespace level makes sense.
   
   The `replicateSubscriptionState` follows hierarchy topic policies, with the difference being that we only apply topic/namespace/broker policy to subscription when the consumer level is false.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the broker, namespace and topic levels [pulsar]

Posted by "nodece (via GitHub)" <gi...@apache.org>.
nodece commented on code in PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#discussion_r1965133986


##########
pip/pip-398.md:
##########
@@ -0,0 +1,245 @@
+# PIP-398: Subscription replication on the broker, namespace and topic levels
+
+# Background knowledge
+
+https://github.com/apache/pulsar/pull/4299 introduces the subscription replication feature on the consumer level:
+
+```java
+Consumer<byte[]> consumer = pulsarClient.newConsumer().topic("topic").replicateSubscriptionState(false/*true*/)
+        .subscriptionName("sub").subscribe();
+```
+
+While this provides flexibility, it introduces overhead in managing replication for a large number of consumers. Users
+need to manually enable the `replicateSubscriptionState` flag for each consumer, which can become cumbersome in
+large-scale deployments.
+
+# Motivation
+
+The key motivation behind this PIP is to simplify subscription replication configuration, especially in failover
+scenarios. When a main cluster goes down and a backup cluster is activated, ensuring that subscription states are
+consistently replicated across clusters is critical for failover scenarios. By extending the replication configuration
+to the broker, namespace and topic levels, the system reduces the need for explicit consumer-level configuration.
+
+# Goals
+
+## In Scope
+
+The PIP aims to provide management of subscription replication at the broker, namespace and topic levels using the
+Pulsar Admin CLI and API.
+
+# High Level Design
+
+Introduces the `replicateSubscriptionState` configuration to enabling subscription replication on the broker, namespace
+and topic levels, when enabled, all consumers under the broker/namespace/topic will automatically replicate their
+subscription states to remote clusters.
+
+The priority for the subscription replication configuration is as follows:
+
+- consumer level > topic level > namespace level > broker level.
+- If `replicateSubscriptionState` is set at the consumer level, configurations at the topic, namespace, and broker levels are
+  ignored.
+- If set at the topic level, the namespace-level configuration is ignored.
+- If set at the namespace level, the broker-level configuration is ignored.
+
+# Detailed Design
+
+## Design & Implementation Details
+
+### Broker level
+
+Add the field `Boolean replicate_subscriptions_state` to the `org.apache.pulsar.broker.ServiceConfiguration` class
+to control subscription replication at the broker level:
+```java
+public class ServiceConfiguration implements PulsarConfiguration {
+    @FieldContext(
+            category = CATEGORY_SERVER,
+            required = false,
+            dynamic = true,
+            doc = "The default value for replicating subscription state."
+    )
+    private Boolean replicate_subscriptions_state;
+}
+```
+
+### Namespace level
+
+1. Add the field `Boolean replicate_subscriptions_state` to the `org.apache.pulsar.common.policies.data.Policies` class
+   to control subscription replication at the namespace level:
+    ```java
+    public class Policies {
+        @SuppressWarnings("checkstyle:MemberName")
+        public Boolean replicate_subscription_state;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.Namespaces` interface:
+     ```java
+     public interface Namespaces {
+         void setReplicateSubscriptionState(String namespace, Boolean enabled) throws PulsarAdminException;
+         CompletableFuture<Void> setReplicateSubscriptionStateAsync(String namespace, Boolean enabled);
+         Boolean getReplicateSubscriptionState(String namespace) throws PulsarAdminException;
+         CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String namespace);
+     }
+     ```
+3. Implement the management methods in the `org.apache.pulsar.client.admin.internal.NamespacesImpl` class.
+
+### Topic level
+
+1. Add the field `Boolean replicateSubscriptionState` to the `org.apache.pulsar.common.policies.data.TopicPolicies`
+   class to enable subscription replication at the topic level:
+   ```java
+    public class TopicPolicies {
+         public Boolean replicateSubscriptionState;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.TopicPolicies` interface:
+   ```java
+       public interface TopicPolicies { 
+           void setReplicateSubscriptionState(String topic, Boolean enabled) throws PulsarAdminException;
+           CompletableFuture<Void> setReplicateSubscriptionStateAsync(String topic, Boolean enabled);
+           Boolean getReplicateSubscriptionState(String topic, boolean applied) throws PulsarAdminException;
+           CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String topic, boolean applied);
+       }
+   ```
+3. Implement the management methods in the `org.apache.pulsar.client.admin.internal.TopicPoliciesImpl` class.
+
+### Consumer level
+
+No changes. When the consumer with `replicateSubscriptionState=true`, the old/new subscription will be snapshot. If `false`, no operation will
+be performed.
+
+### Subscription replication applied
+
+When the subscription replication is changed on the broker, namespace or topic level, the subscription replication will
+be applied to all consumers under the broker/namespace/topic.
+
+We exclusively consider the `true` case for replication. If the consumer level is set to `false`, the
+replication configuration defined at the topic, namespace, or broker level will be applied to the subscription.
+
+There is a special case here, if the user intentionally sets `false` at the consumer level, and then set `true` at the
+topic/ns/broker level, this may disrupt the user's behavior. This way, we can minimize changes to the Pulsar public API
+as much as possible.

Review Comment:
   The user expected to disable subscription replication, however, found that the feature had been enabled.
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the broker, namespace and topic levels [pulsar]

Posted by "merlimat (via GitHub)" <gi...@apache.org>.
merlimat closed pull request #23770: [improve][pip] PIP-398: Subscription replication on the broker, namespace and topic levels
URL: https://github.com/apache/pulsar/pull/23770


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the namespace and topic levels [pulsar]

Posted by "yyj8 (via GitHub)" <gi...@apache.org>.
yyj8 commented on PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#issuecomment-2559532882

   > > > @nodece would it be possible to cover #23769 feature too? That enables subscription replication for all subscriptions at the broker level if I understand it correctly. Could you work together with @yyj8 to cover that?
   > > 
   > > 
   > > #23769 aims to overwrite the subscription replication state from the consumer, which is different from this PIP. Actually, I also need an overwrite feature, and then the users use the admin API to manage the subscription replication.
   > > The downstream users sometimes enable/disable the subscription replication on the consumer, this lead to the namespace or topic level is ignored.
   > > We can also merge #23769 to this PIP. What's your opinion? @lhotari @yyj8
   > 
   > @nodece @yyj8 Please work together to combine both cases since they are both valuable and useful. I think it would make sense to combine the work in a single PIP to ensure that we achieve consistency across the changes.
   
   
   
   > > > @nodece would it be possible to cover #23769 feature too? That enables subscription replication for all subscriptions at the broker level if I understand it correctly. Could you work together with @yyj8 to cover that?
   > > 
   > > 
   > > #23769 aims to overwrite the subscription replication state from the consumer, which is different from this PIP. Actually, I also need an overwrite feature, and then the users use the admin API to manage the subscription replication.
   > > The downstream users sometimes enable/disable the subscription replication on the consumer, this lead to the namespace or topic level is ignored.
   > > We can also merge #23769 to this PIP. What's your opinion? @lhotari @yyj8
   > 
   > @nodece @yyj8 Please work together to combine both cases since they are both valuable and useful. I think it would make sense to combine the work in a single PIP to ensure that we achieve consistency across the changes.
   
   This is a great idea and suggestion. Can we consider a strategy that divides into three dimensions: cluster dimension, namespace dimension, and topic dimension.
   (1)PR https://github.com/apache/pulsar/pull/23769 As a cluster dimension strategy, merge it into this PIP https://github.com/apache/pulsar/pull/23770;
   (2)PR https://github.com/apache/pulsar/pull/23769 add the function of dynamically configuring parameter `replicateAllSubscriptionState` values through the command `bin/pulse admin brokers update-dynamic-config 
    --config replicateAllSubscriptionState --value true/false`
   
   @nodece @lhotari Can you see if this method is feasible?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the broker, namespace and topic levels [pulsar]

Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#issuecomment-2665396306

   The term `replicateSubscriptionState` is ambiguous (it's not introduced in this PIP). It's unclear whether it refers to "replicating the subscription state" or represents "the state of replicating subscriptions". This demonstrates why putting more thought into naming concepts could be helpful.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the broker, namespace and topic levels [pulsar]

Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on code in PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#discussion_r1959550115


##########
pip/pip-398.md:
##########
@@ -0,0 +1,245 @@
+# PIP-398: Subscription replication on the broker, namespace and topic levels
+
+# Background knowledge
+
+https://github.com/apache/pulsar/pull/4299 introduces the subscription replication feature on the consumer level:
+
+```java
+Consumer<byte[]> consumer = pulsarClient.newConsumer().topic("topic").replicateSubscriptionState(false/*true*/)
+        .subscriptionName("sub").subscribe();
+```
+
+While this provides flexibility, it introduces overhead in managing replication for a large number of consumers. Users
+need to manually enable the `replicateSubscriptionState` flag for each consumer, which can become cumbersome in
+large-scale deployments.
+
+# Motivation
+
+The key motivation behind this PIP is to simplify subscription replication configuration, especially in failover
+scenarios. When a main cluster goes down and a backup cluster is activated, ensuring that subscription states are
+consistently replicated across clusters is critical for failover scenarios. By extending the replication configuration
+to the broker, namespace and topic levels, the system reduces the need for explicit consumer-level configuration.
+
+# Goals
+
+## In Scope
+
+The PIP aims to provide management of subscription replication at the broker, namespace and topic levels using the
+Pulsar Admin CLI and API.
+
+# High Level Design
+
+Introduces a configuration used for enabling subscription replication on the broker, namespace and topic levels, when
+enabled, all consumers under the broker/namespace/topic will automatically replicate their subscription states to remote
+clusters.
+
+The priority for the subscription replication configuration is as follows:
+
+- consumer level > topic level > namespace level > broker level.
+- If `replicateSubscriptionState` is set at the consumer level, configurations at the topic, namespace, and broker levels are
+  ignored.
+- If set at the topic level, the namespace-level configuration is ignored.
+- If set at the namespace level, the broker-level configuration is ignored.
+
+# Detailed Design
+
+## Design & Implementation Details
+
+### Broker level
+
+Add the field `Boolean replicate_subscriptions_state` to the `org.apache.pulsar.broker.ServiceConfiguration` class
+to control subscription replication at the broker level:
+```java
+public class ServiceConfiguration implements PulsarConfiguration {
+    @FieldContext(
+            category = CATEGORY_SERVER,
+            required = false,
+            dynamic = true,
+            doc = "The default value for replicating subscription state."
+    )
+    private Boolean replicate_subscriptions_state;
+}
+```
+
+### Namespace level
+
+1. Add the field `Boolean replicate_subscriptions_state` to the `org.apache.pulsar.common.policies.data.Policies` class
+   to control subscription replication at the namespace level:
+    ```java
+    public class Policies {
+        @SuppressWarnings("checkstyle:MemberName")
+        public Boolean replicate_subscription_state;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.Namespaces` interface:
+     ```java
+     public interface Namespaces {
+         void setReplicateSubscriptionState(String namespace, Boolean enabled) throws PulsarAdminException;
+         CompletableFuture<Void> setReplicateSubscriptionStateAsync(String namespace, Boolean enabled);
+         Boolean getReplicateSubscriptionState(String namespace) throws PulsarAdminException;
+         CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String namespace);
+     }
+     ```
+3. Implement the management methods in the `org.apache.pulsar.client.admin.internal.NamespacesImpl` class.
+
+### Topic level
+
+1. Add the field `Boolean replicateSubscriptionState` to the `org.apache.pulsar.common.policies.data.TopicPolicies`
+   class to enable subscription replication at the topic level:
+   ```java
+    public class TopicPolicies {
+         public Boolean replicateSubscriptionState;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.TopicPolicies` interface:
+   ```java
+       public interface TopicPolicies { 
+           void setReplicateSubscriptionState(String topic, Boolean enabled) throws PulsarAdminException;
+           CompletableFuture<Void> setReplicateSubscriptionStateAsync(String topic, Boolean enabled);
+           Boolean getReplicateSubscriptionState(String topic, boolean applied) throws PulsarAdminException;
+           CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String topic, boolean applied);
+       }
+   ```
+3. Implement the management methods in the `org.apache.pulsar.client.admin.internal.TopicPoliciesImpl` class.
+
+### Consumer level
+
+No changes. When the consumer with `replicateSubscriptionState=true`, the old/new subscription will be snapshot. If `false`, no operation will
+be performed.
+
+### Subscription replication applied
+
+When the subscription replication is changed on the broker, namespace or topic level, the subscription replication will
+be applied to all consumers under the broker/namespace/topic.
+
+We exclusively consider the `true` case for replication. If the consumer level is set to `false`, the
+replication configuration defined at the topic, namespace, or broker level will be applied to the subscription.
+
+There is a special case here, if the user intentionally sets `false` at the consumer level, and then set `true` at the
+topic/ns/broker level, this may disrupt the user's behavior. This way, we can minimize changes to the Pulsar public API
+as much as possible.
+
+## Public-facing Changes
+
+### Public API
+
+##### Namespace level
+
+- `/{tenant}/{namespace}/replicateSubscriptionState`: enable/disable/remove the subscription replication on the
+  namespace level.
+    - Method: `POST`
+    - Content-Type: `application/json`
+    - Body:
+        - true
+        - false
+        - null
+- `GET /{tenant}/{namespace}/replicateSubscriptionState` to get subscription replication configuration on the namespace
+  level.
+    - Method: `GET`
+    - Response:
+        - true
+        - false
+        - null

Review Comment:
   Just wondering if this would be more of a namespace policy matter at the namespace level



##########
pip/pip-398.md:
##########
@@ -0,0 +1,245 @@
+# PIP-398: Subscription replication on the broker, namespace and topic levels
+
+# Background knowledge
+
+https://github.com/apache/pulsar/pull/4299 introduces the subscription replication feature on the consumer level:
+
+```java
+Consumer<byte[]> consumer = pulsarClient.newConsumer().topic("topic").replicateSubscriptionState(false/*true*/)
+        .subscriptionName("sub").subscribe();
+```
+
+While this provides flexibility, it introduces overhead in managing replication for a large number of consumers. Users
+need to manually enable the `replicateSubscriptionState` flag for each consumer, which can become cumbersome in
+large-scale deployments.
+
+# Motivation
+
+The key motivation behind this PIP is to simplify subscription replication configuration, especially in failover
+scenarios. When a main cluster goes down and a backup cluster is activated, ensuring that subscription states are
+consistently replicated across clusters is critical for failover scenarios. By extending the replication configuration
+to the broker, namespace and topic levels, the system reduces the need for explicit consumer-level configuration.
+
+# Goals
+
+## In Scope
+
+The PIP aims to provide management of subscription replication at the broker, namespace and topic levels using the
+Pulsar Admin CLI and API.
+
+# High Level Design
+
+Introduces a configuration used for enabling subscription replication on the broker, namespace and topic levels, when
+enabled, all consumers under the broker/namespace/topic will automatically replicate their subscription states to remote
+clusters.
+
+The priority for the subscription replication configuration is as follows:
+
+- consumer level > topic level > namespace level > broker level.
+- If `replicateSubscriptionState` is set at the consumer level, configurations at the topic, namespace, and broker levels are
+  ignored.
+- If set at the topic level, the namespace-level configuration is ignored.
+- If set at the namespace level, the broker-level configuration is ignored.

Review Comment:
   I think that there could be a need to override any value set on the consumer side. I also added some comments that controlling the behavior at namespace and consumer level should perhaps be a policy concern instead of creating a separate concept for handling the setting.
   
   If the setting at namespace and topic level would be more than just a true/false. It would be possible to add a separate setting that would allow overriding the consumer level setting.
   



##########
pip/pip-398.md:
##########
@@ -0,0 +1,245 @@
+# PIP-398: Subscription replication on the broker, namespace and topic levels
+
+# Background knowledge
+
+https://github.com/apache/pulsar/pull/4299 introduces the subscription replication feature on the consumer level:
+
+```java
+Consumer<byte[]> consumer = pulsarClient.newConsumer().topic("topic").replicateSubscriptionState(false/*true*/)
+        .subscriptionName("sub").subscribe();
+```
+
+While this provides flexibility, it introduces overhead in managing replication for a large number of consumers. Users
+need to manually enable the `replicateSubscriptionState` flag for each consumer, which can become cumbersome in
+large-scale deployments.
+
+# Motivation
+
+The key motivation behind this PIP is to simplify subscription replication configuration, especially in failover
+scenarios. When a main cluster goes down and a backup cluster is activated, ensuring that subscription states are
+consistently replicated across clusters is critical for failover scenarios. By extending the replication configuration
+to the broker, namespace and topic levels, the system reduces the need for explicit consumer-level configuration.
+
+# Goals
+
+## In Scope
+
+The PIP aims to provide management of subscription replication at the broker, namespace and topic levels using the
+Pulsar Admin CLI and API.
+
+# High Level Design
+
+Introduces a configuration used for enabling subscription replication on the broker, namespace and topic levels, when
+enabled, all consumers under the broker/namespace/topic will automatically replicate their subscription states to remote
+clusters.
+
+The priority for the subscription replication configuration is as follows:
+
+- consumer level > topic level > namespace level > broker level.
+- If `replicateSubscriptionState` is set at the consumer level, configurations at the topic, namespace, and broker levels are
+  ignored.
+- If set at the topic level, the namespace-level configuration is ignored.
+- If set at the namespace level, the broker-level configuration is ignored.
+
+# Detailed Design
+
+## Design & Implementation Details
+
+### Broker level
+
+Add the field `Boolean replicate_subscriptions_state` to the `org.apache.pulsar.broker.ServiceConfiguration` class
+to control subscription replication at the broker level:
+```java
+public class ServiceConfiguration implements PulsarConfiguration {
+    @FieldContext(
+            category = CATEGORY_SERVER,
+            required = false,
+            dynamic = true,
+            doc = "The default value for replicating subscription state."
+    )
+    private Boolean replicate_subscriptions_state;
+}
+```
+
+### Namespace level
+
+1. Add the field `Boolean replicate_subscriptions_state` to the `org.apache.pulsar.common.policies.data.Policies` class
+   to control subscription replication at the namespace level:
+    ```java
+    public class Policies {
+        @SuppressWarnings("checkstyle:MemberName")
+        public Boolean replicate_subscription_state;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.Namespaces` interface:
+     ```java
+     public interface Namespaces {
+         void setReplicateSubscriptionState(String namespace, Boolean enabled) throws PulsarAdminException;
+         CompletableFuture<Void> setReplicateSubscriptionStateAsync(String namespace, Boolean enabled);
+         Boolean getReplicateSubscriptionState(String namespace) throws PulsarAdminException;
+         CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String namespace);
+     }
+     ```
+3. Implement the management methods in the `org.apache.pulsar.client.admin.internal.NamespacesImpl` class.
+
+### Topic level
+
+1. Add the field `Boolean replicateSubscriptionState` to the `org.apache.pulsar.common.policies.data.TopicPolicies`
+   class to enable subscription replication at the topic level:
+   ```java
+    public class TopicPolicies {
+         public Boolean replicateSubscriptionState;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.TopicPolicies` interface:
+   ```java
+       public interface TopicPolicies { 
+           void setReplicateSubscriptionState(String topic, Boolean enabled) throws PulsarAdminException;
+           CompletableFuture<Void> setReplicateSubscriptionStateAsync(String topic, Boolean enabled);
+           Boolean getReplicateSubscriptionState(String topic, boolean applied) throws PulsarAdminException;
+           CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String topic, boolean applied);
+       }
+   ```
+3. Implement the management methods in the `org.apache.pulsar.client.admin.internal.TopicPoliciesImpl` class.
+
+### Consumer level
+
+No changes. When the consumer with `replicateSubscriptionState=true`, the old/new subscription will be snapshot. If `false`, no operation will
+be performed.
+
+### Subscription replication applied
+
+When the subscription replication is changed on the broker, namespace or topic level, the subscription replication will
+be applied to all consumers under the broker/namespace/topic.
+
+We exclusively consider the `true` case for replication. If the consumer level is set to `false`, the
+replication configuration defined at the topic, namespace, or broker level will be applied to the subscription.
+
+There is a special case here, if the user intentionally sets `false` at the consumer level, and then set `true` at the
+topic/ns/broker level, this may disrupt the user's behavior. This way, we can minimize changes to the Pulsar public API
+as much as possible.
+
+## Public-facing Changes
+
+### Public API
+
+##### Namespace level
+
+- `/{tenant}/{namespace}/replicateSubscriptionState`: enable/disable/remove the subscription replication on the
+  namespace level.
+    - Method: `POST`
+    - Content-Type: `application/json`
+    - Body:
+        - true
+        - false
+        - null
+- `GET /{tenant}/{namespace}/replicateSubscriptionState` to get subscription replication configuration on the namespace
+  level.
+    - Method: `GET`
+    - Response:
+        - true
+        - false
+        - null
+
+##### Topic level
+
+- `/{tenant}/{namespace}/{topic}/replicateSubscriptionState`: enable/disable/remove the subscription replication on the
+  topic level.
+    - Method: `POST`
+    - Content-Type: `application/json`
+    - Body:
+        - true
+        - false
+        - null
+- `/{tenant}/{namespace}/{topic}/replicateSubscriptionState` to get subscription replication configuration on the topic
+  level.
+    - Method: `GET`
+    - Parameters:
+        - `applied=true`: get the applied subscription replication configuration, if topic is not set, return the
+          namespace level configuration.
+        - `applied=false`: get the applied subscription replication configuration, if topic is set, return the topic
+          level configuration.
+    - Response:
+        - true
+        - false
+        - null

Review Comment:
   It feels that this should be part of the topic policy. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the broker, namespace and topic levels [pulsar]

Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#issuecomment-2673976972

   > The `replicateSubscriptionState` follows hierarchy topic policies, with the difference being that we only apply topic/namespace/broker policy to subscription when the consumer level is false.
   
   @nodece Yes, I understand that. Isn't there a need to override any client application consumer level setting with topic or namespace level policies? What if Pulsar administrator would explicitly want to disallow or force using replicated subscriptions for a particular topic or namespace while geo-replication is enabled?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the broker, namespace and topic levels [pulsar]

Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#discussion_r2917091133


##########
pip/pip-398.md:
##########
@@ -0,0 +1,291 @@
+# PIP-398: Subscription replication on the broker, namespace and topic levels
+
+# Background knowledge
+
+https://github.com/apache/pulsar/pull/4299 introduces the subscription replication feature on the consumer level:
+
+```java
+Consumer<byte[]> consumer = pulsarClient.newConsumer().topic("topic").replicateSubscriptionState(true /* false */)
+        .subscriptionName("sub").subscribe();
+```
+
+While this provides flexibility, it introduces overhead in managing replication for a large number of consumers. Users
+need to manually enable the `replicateSubscriptionState` flag for each consumer, which can become cumbersome in
+large-scale deployments.
+
+# Motivation

Review Comment:
   The document uses multiple top-level `#` headings after the title. For consistent Markdown structure (and better TOC rendering), section headings after `# PIP-398: ...` should typically be `## ...` (and deeper sections `### ...`).



##########
pip/pip-398.md:
##########
@@ -0,0 +1,291 @@
+# PIP-398: Subscription replication on the broker, namespace and topic levels
+
+# Background knowledge
+
+https://github.com/apache/pulsar/pull/4299 introduces the subscription replication feature on the consumer level:
+
+```java
+Consumer<byte[]> consumer = pulsarClient.newConsumer().topic("topic").replicateSubscriptionState(true /* false */)
+        .subscriptionName("sub").subscribe();
+```
+
+While this provides flexibility, it introduces overhead in managing replication for a large number of consumers. Users
+need to manually enable the `replicateSubscriptionState` flag for each consumer, which can become cumbersome in
+large-scale deployments.
+
+# Motivation
+
+The key motivation behind this PIP is to simplify subscription replication configuration, especially in failover
+scenarios. When a main cluster goes down and a backup cluster is activated, ensuring that subscription states are
+consistently replicated across clusters is critical for failover scenarios. By extending the replication configuration
+to the broker, namespace and topic levels, the system reduces the need for explicit consumer-level configuration.
+
+# Goals

Review Comment:
   The document uses multiple top-level `#` headings after the title. For consistent Markdown structure (and better TOC rendering), section headings after `# PIP-398: ...` should typically be `## ...` (and deeper sections `### ...`).



##########
pip/pip-398.md:
##########
@@ -0,0 +1,291 @@
+# PIP-398: Subscription replication on the broker, namespace and topic levels
+
+# Background knowledge
+
+https://github.com/apache/pulsar/pull/4299 introduces the subscription replication feature on the consumer level:
+
+```java
+Consumer<byte[]> consumer = pulsarClient.newConsumer().topic("topic").replicateSubscriptionState(true /* false */)
+        .subscriptionName("sub").subscribe();
+```
+
+While this provides flexibility, it introduces overhead in managing replication for a large number of consumers. Users
+need to manually enable the `replicateSubscriptionState` flag for each consumer, which can become cumbersome in
+large-scale deployments.
+
+# Motivation
+
+The key motivation behind this PIP is to simplify subscription replication configuration, especially in failover
+scenarios. When a main cluster goes down and a backup cluster is activated, ensuring that subscription states are
+consistently replicated across clusters is critical for failover scenarios. By extending the replication configuration
+to the broker, namespace and topic levels, the system reduces the need for explicit consumer-level configuration.
+
+# Goals
+
+## In Scope
+
+- Provide management of subscription replication at the broker, namespace and topic levels using the Pulsar Admin CLI
+  and API.
+- Define the priority order for subscription replication configuration across different levels.
+
+## Out of Scope
+
+- Changes to the existing consumer-level `replicateSubscriptionState` configuration.
+- Changes to the subscription replication mechanism itself (how states are replicated across clusters).
+
+# High Level Design
+
+This PIP introduces the `replicateSubscriptionState` configuration at the **broker, namespace, and topic levels** to
+simplify the management of subscription replication.
+
+With this proposal, administrators can configure subscription replication at higher levels so that consumers
+automatically inherit the effective configuration.
+
+# Enable-only Semantics & Effective Logic
+
+## Enable-only Semantics
+
+Subscription replication policies follow Pulsar's hierarchical configuration model:
+
+```
+Topic > Namespace > Broker
+```
+
+The **highest level with a configured value determines the effective policy**; lower levels are ignored once a
+higher-level configuration exists.
+
+* **Broker, Namespace, and Topic levels** follow **enable-only semantics**:
+    * `true` → enables subscription replication.
+    * `false` or `null` → do **not disable** existing replicated subscriptions.
+
+* **Consumer configuration** participates in the evaluation but is **not part of the policy hierarchy**:
+    * `true` → enables replication.
+    * `false` → does **not automatically disable** replication.
+    * `null` → inherits the effective configuration from higher-level policies.
+
+Explicitly **disabling replication** for a subscription is only possible via the **Pulsar Admin API** targeting that
+subscription.
+
+## Effective Replication Logic
+
+The broker determines the replication state for each subscription as follows:
+
+1. Determine the **effective policy** using Pulsar's precedence: `Topic > Namespace > Broker`.
+
+2. If the **effective policy is `true`**, the subscription is marked as replicated.
+
+3. If the effective policy is not `true`, the broker evaluates the **consumer configuration**:
+    * `true` → enables replication
+    * `false` or `null` → no change to the existing subscription state
+
+4. A subscription can be explicitly disabled via **Pulsar Admin**, but this succeeds **only if all higher-level policies
+   ** (broker, namespace, topic) are `false` or `null`.

Review Comment:
   The bold formatting is broken due to an unmatched/interrupted `**` across lines (line 82 starts with `**`). Rephrase so the emphasis markers are balanced (e.g., keep the bold phrase on one line or close/open `**` correctly).
   ```suggestion
      (broker, namespace, topic) are `false` or `null`.**
   ```



##########
pip/pip-398.md:
##########
@@ -0,0 +1,291 @@
+# PIP-398: Subscription replication on the broker, namespace and topic levels
+
+# Background knowledge
+
+https://github.com/apache/pulsar/pull/4299 introduces the subscription replication feature on the consumer level:
+
+```java
+Consumer<byte[]> consumer = pulsarClient.newConsumer().topic("topic").replicateSubscriptionState(true /* false */)
+        .subscriptionName("sub").subscribe();
+```
+
+While this provides flexibility, it introduces overhead in managing replication for a large number of consumers. Users
+need to manually enable the `replicateSubscriptionState` flag for each consumer, which can become cumbersome in
+large-scale deployments.
+
+# Motivation
+
+The key motivation behind this PIP is to simplify subscription replication configuration, especially in failover
+scenarios. When a main cluster goes down and a backup cluster is activated, ensuring that subscription states are
+consistently replicated across clusters is critical for failover scenarios. By extending the replication configuration
+to the broker, namespace and topic levels, the system reduces the need for explicit consumer-level configuration.
+
+# Goals
+
+## In Scope
+
+- Provide management of subscription replication at the broker, namespace and topic levels using the Pulsar Admin CLI
+  and API.
+- Define the priority order for subscription replication configuration across different levels.
+
+## Out of Scope
+
+- Changes to the existing consumer-level `replicateSubscriptionState` configuration.
+- Changes to the subscription replication mechanism itself (how states are replicated across clusters).
+
+# High Level Design
+
+This PIP introduces the `replicateSubscriptionState` configuration at the **broker, namespace, and topic levels** to
+simplify the management of subscription replication.
+
+With this proposal, administrators can configure subscription replication at higher levels so that consumers
+automatically inherit the effective configuration.
+
+# Enable-only Semantics & Effective Logic
+
+## Enable-only Semantics
+
+Subscription replication policies follow Pulsar's hierarchical configuration model:
+
+```
+Topic > Namespace > Broker
+```
+
+The **highest level with a configured value determines the effective policy**; lower levels are ignored once a
+higher-level configuration exists.
+
+* **Broker, Namespace, and Topic levels** follow **enable-only semantics**:
+    * `true` → enables subscription replication.
+    * `false` or `null` → do **not disable** existing replicated subscriptions.
+
+* **Consumer configuration** participates in the evaluation but is **not part of the policy hierarchy**:
+    * `true` → enables replication.
+    * `false` → does **not automatically disable** replication.
+    * `null` → inherits the effective configuration from higher-level policies.
+
+Explicitly **disabling replication** for a subscription is only possible via the **Pulsar Admin API** targeting that
+subscription.
+
+## Effective Replication Logic
+
+The broker determines the replication state for each subscription as follows:
+
+1. Determine the **effective policy** using Pulsar's precedence: `Topic > Namespace > Broker`.
+
+2. If the **effective policy is `true`**, the subscription is marked as replicated.
+
+3. If the effective policy is not `true`, the broker evaluates the **consumer configuration**:
+    * `true` → enables replication
+    * `false` or `null` → no change to the existing subscription state
+
+4. A subscription can be explicitly disabled via **Pulsar Admin**, but this succeeds **only if all higher-level policies
+   ** (broker, namespace, topic) are `false` or `null`.
+
+   If a higher-level policy later evaluates to `true` (e.g., due to topic reload or namespace/broker policy change),
+   the broker will reapply the effective policy and mark the subscription as replicated again.
+
+### Design Consistency with Consumer-level Logic
+
+This design builds upon the existing **consumer-level logic**, which already follows **enable-only semantics**:
+
+```java
+// https://github.com/apache/pulsar/blob/8798a4662300dfa3d330dbb0277073874cc2d86e/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentTopic.java#L1153-L1156
+if (replicated != null && replicated && !subscription.isReplicated()) {
+    subscription.setReplicated(true);
+}
+```
+
+Key points:
+
+* **Consumer-level default changed** from `false` to `null` ([PR #23757](https://github.com/apache/pulsar/pull/23757)).
+  * Combined with the multi-level policy evaluation, this ensures **compatibility across all client languages and versions**:
+      * Older clients defaulting to `false`
+      * Newer clients defaulting to `null`
+  * Both behave correctly with **broker, namespace, and topic-level policies**.
+
+# Detailed Design
+
+## Design & Implementation Details
+
+### Broker level
+
+Add the field `replicate_subscriptions_state` to the `org.apache.pulsar.broker.ServiceConfiguration` class
+to control subscription replication at the broker level:
+
+```java
+public class ServiceConfiguration implements PulsarConfiguration {
+    @FieldContext(
+            category = CATEGORY_SERVER,
+            required = false,
+            dynamic = false,
+            doc = "The default value for replicating subscription state."
+    )
+    private Boolean replicate_subscriptions_state;
+}
+```
+
+### Namespace level
+
+1. Add the field `replicate_subscriptions_state` to the `org.apache.pulsar.common.policies.data.Policies` class
+   to control subscription replication at the namespace level:
+    ```java
+    public class Policies {
+        @SuppressWarnings("checkstyle:MemberName")
+        public Boolean replicate_subscription_state;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.Namespaces` interface:
+     ```java
+     public interface Namespaces {
+         void setReplicateSubscriptionState(String namespace, boolean enabled) throws PulsarAdminException;
+         CompletableFuture<Void> setReplicateSubscriptionStateAsync(String namespace, boolean enabled);
+         Boolean getReplicateSubscriptionState(String namespace) throws PulsarAdminException;
+         CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String namespace);
+         void removeReplicateSubscriptionState(String namespace) throws PulsarAdminException;
+         CompletableFuture<Void> removeReplicateSubscriptionStateAsync(String namespace);
+     }
+     ```
+3. Implement the management methods in the `org.apache.pulsar.client.admin.internal.NamespacesImpl` class.
+
+### Topic level
+
+1. Add the field `Boolean replicateSubscriptionState` to the `org.apache.pulsar.common.policies.data.TopicPolicies`
+   class to enable subscription replication at the topic level:
+   ```java
+    public class TopicPolicies {
+         public Boolean replicateSubscriptionState;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.TopicPolicies` interface:
+   ```java
+       public interface TopicPolicies {
+           void setReplicateSubscriptionState(String topic, boolean enabled) throws PulsarAdminException;
+           CompletableFuture<Void> setReplicateSubscriptionStateAsync(String topic, boolean enabled);
+           Boolean getReplicateSubscriptionState(String topic, boolean applied) throws PulsarAdminException;
+           CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String topic, boolean applied);
+           void removeReplicateSubscriptionState(String topic) throws PulsarAdminException;
+           CompletableFuture<Void> removeReplicateSubscriptionStateAsync(String topic);
+       }
+   ```
+3. Implement the management methods in the `org.apache.pulsar.client.admin.internal.TopicPoliciesImpl` class.
+
+### Consumer level
+
+No changes. When the consumer with `replicateSubscriptionState=true`, the subscription will be snapshot. If `false`, no
+operation will be performed.
+
+## Public-facing Changes
+
+### Public API
+
+> **Note:** Setting `false` at namespace or topic level is accepted but not applied (enable-only semantics). Use the
+`DELETE` endpoint or `remove-` command to clear the configuration instead.

Review Comment:
   There’s a semantic inconsistency: earlier text implies `false` is a configured value (which would participate in precedence and potentially override lower levels), but later the note says `false` is 'accepted but not applied' (suggesting it behaves like 'unset'). Please clarify whether `false` is stored and affects precedence (even if it doesn’t actively disable existing subscriptions), or whether it should be treated as equivalent to `null`/not-configured and therefore should not override a lower-level `true`.



##########
pip/pip-398.md:
##########
@@ -0,0 +1,291 @@
+# PIP-398: Subscription replication on the broker, namespace and topic levels
+
+# Background knowledge
+
+https://github.com/apache/pulsar/pull/4299 introduces the subscription replication feature on the consumer level:
+
+```java
+Consumer<byte[]> consumer = pulsarClient.newConsumer().topic("topic").replicateSubscriptionState(true /* false */)
+        .subscriptionName("sub").subscribe();
+```
+
+While this provides flexibility, it introduces overhead in managing replication for a large number of consumers. Users
+need to manually enable the `replicateSubscriptionState` flag for each consumer, which can become cumbersome in
+large-scale deployments.
+
+# Motivation
+
+The key motivation behind this PIP is to simplify subscription replication configuration, especially in failover
+scenarios. When a main cluster goes down and a backup cluster is activated, ensuring that subscription states are
+consistently replicated across clusters is critical for failover scenarios. By extending the replication configuration
+to the broker, namespace and topic levels, the system reduces the need for explicit consumer-level configuration.
+
+# Goals
+
+## In Scope
+
+- Provide management of subscription replication at the broker, namespace and topic levels using the Pulsar Admin CLI
+  and API.
+- Define the priority order for subscription replication configuration across different levels.
+
+## Out of Scope
+
+- Changes to the existing consumer-level `replicateSubscriptionState` configuration.
+- Changes to the subscription replication mechanism itself (how states are replicated across clusters).
+
+# High Level Design
+
+This PIP introduces the `replicateSubscriptionState` configuration at the **broker, namespace, and topic levels** to
+simplify the management of subscription replication.
+
+With this proposal, administrators can configure subscription replication at higher levels so that consumers
+automatically inherit the effective configuration.
+
+# Enable-only Semantics & Effective Logic
+
+## Enable-only Semantics
+
+Subscription replication policies follow Pulsar's hierarchical configuration model:
+
+```
+Topic > Namespace > Broker
+```
+
+The **highest level with a configured value determines the effective policy**; lower levels are ignored once a
+higher-level configuration exists.
+
+* **Broker, Namespace, and Topic levels** follow **enable-only semantics**:
+    * `true` → enables subscription replication.
+    * `false` or `null` → do **not disable** existing replicated subscriptions.
+
+* **Consumer configuration** participates in the evaluation but is **not part of the policy hierarchy**:
+    * `true` → enables replication.
+    * `false` → does **not automatically disable** replication.
+    * `null` → inherits the effective configuration from higher-level policies.
+
+Explicitly **disabling replication** for a subscription is only possible via the **Pulsar Admin API** targeting that
+subscription.
+
+## Effective Replication Logic
+
+The broker determines the replication state for each subscription as follows:
+
+1. Determine the **effective policy** using Pulsar's precedence: `Topic > Namespace > Broker`.
+
+2. If the **effective policy is `true`**, the subscription is marked as replicated.
+
+3. If the effective policy is not `true`, the broker evaluates the **consumer configuration**:
+    * `true` → enables replication
+    * `false` or `null` → no change to the existing subscription state
+
+4. A subscription can be explicitly disabled via **Pulsar Admin**, but this succeeds **only if all higher-level policies
+   ** (broker, namespace, topic) are `false` or `null`.
+
+   If a higher-level policy later evaluates to `true` (e.g., due to topic reload or namespace/broker policy change),
+   the broker will reapply the effective policy and mark the subscription as replicated again.
+
+### Design Consistency with Consumer-level Logic
+
+This design builds upon the existing **consumer-level logic**, which already follows **enable-only semantics**:
+
+```java
+// https://github.com/apache/pulsar/blob/8798a4662300dfa3d330dbb0277073874cc2d86e/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentTopic.java#L1153-L1156
+if (replicated != null && replicated && !subscription.isReplicated()) {
+    subscription.setReplicated(true);
+}
+```
+
+Key points:
+
+* **Consumer-level default changed** from `false` to `null` ([PR #23757](https://github.com/apache/pulsar/pull/23757)).
+  * Combined with the multi-level policy evaluation, this ensures **compatibility across all client languages and versions**:
+      * Older clients defaulting to `false`
+      * Newer clients defaulting to `null`
+  * Both behave correctly with **broker, namespace, and topic-level policies**.
+
+# Detailed Design
+
+## Design & Implementation Details
+
+### Broker level
+
+Add the field `replicate_subscriptions_state` to the `org.apache.pulsar.broker.ServiceConfiguration` class
+to control subscription replication at the broker level:
+
+```java
+public class ServiceConfiguration implements PulsarConfiguration {
+    @FieldContext(
+            category = CATEGORY_SERVER,
+            required = false,
+            dynamic = false,
+            doc = "The default value for replicating subscription state."
+    )
+    private Boolean replicate_subscriptions_state;
+}
+```
+
+### Namespace level
+
+1. Add the field `replicate_subscriptions_state` to the `org.apache.pulsar.common.policies.data.Policies` class

Review Comment:
   The proposed config/property name is inconsistent: the text says `replicate_subscriptions_state` (plural `subscriptions`) while the `Policies` snippet uses `replicate_subscription_state` (singular `subscription`). This will be confusing for implementers and API consumers; align the name consistently across broker/namespace/topic representations (and ensure the chosen name matches the intended config key/public API).



##########
pip/pip-398.md:
##########
@@ -0,0 +1,291 @@
+# PIP-398: Subscription replication on the broker, namespace and topic levels
+
+# Background knowledge
+
+https://github.com/apache/pulsar/pull/4299 introduces the subscription replication feature on the consumer level:
+
+```java
+Consumer<byte[]> consumer = pulsarClient.newConsumer().topic("topic").replicateSubscriptionState(true /* false */)
+        .subscriptionName("sub").subscribe();
+```
+
+While this provides flexibility, it introduces overhead in managing replication for a large number of consumers. Users
+need to manually enable the `replicateSubscriptionState` flag for each consumer, which can become cumbersome in
+large-scale deployments.
+
+# Motivation
+
+The key motivation behind this PIP is to simplify subscription replication configuration, especially in failover
+scenarios. When a main cluster goes down and a backup cluster is activated, ensuring that subscription states are
+consistently replicated across clusters is critical for failover scenarios. By extending the replication configuration
+to the broker, namespace and topic levels, the system reduces the need for explicit consumer-level configuration.
+
+# Goals
+
+## In Scope
+
+- Provide management of subscription replication at the broker, namespace and topic levels using the Pulsar Admin CLI
+  and API.
+- Define the priority order for subscription replication configuration across different levels.
+
+## Out of Scope
+
+- Changes to the existing consumer-level `replicateSubscriptionState` configuration.
+- Changes to the subscription replication mechanism itself (how states are replicated across clusters).
+
+# High Level Design
+
+This PIP introduces the `replicateSubscriptionState` configuration at the **broker, namespace, and topic levels** to
+simplify the management of subscription replication.
+
+With this proposal, administrators can configure subscription replication at higher levels so that consumers
+automatically inherit the effective configuration.
+
+# Enable-only Semantics & Effective Logic
+
+## Enable-only Semantics
+
+Subscription replication policies follow Pulsar's hierarchical configuration model:
+
+```
+Topic > Namespace > Broker
+```
+
+The **highest level with a configured value determines the effective policy**; lower levels are ignored once a
+higher-level configuration exists.
+
+* **Broker, Namespace, and Topic levels** follow **enable-only semantics**:
+    * `true` → enables subscription replication.
+    * `false` or `null` → do **not disable** existing replicated subscriptions.
+
+* **Consumer configuration** participates in the evaluation but is **not part of the policy hierarchy**:
+    * `true` → enables replication.
+    * `false` → does **not automatically disable** replication.
+    * `null` → inherits the effective configuration from higher-level policies.
+
+Explicitly **disabling replication** for a subscription is only possible via the **Pulsar Admin API** targeting that
+subscription.
+
+## Effective Replication Logic
+
+The broker determines the replication state for each subscription as follows:
+
+1. Determine the **effective policy** using Pulsar's precedence: `Topic > Namespace > Broker`.
+
+2. If the **effective policy is `true`**, the subscription is marked as replicated.
+
+3. If the effective policy is not `true`, the broker evaluates the **consumer configuration**:
+    * `true` → enables replication
+    * `false` or `null` → no change to the existing subscription state
+
+4. A subscription can be explicitly disabled via **Pulsar Admin**, but this succeeds **only if all higher-level policies
+   ** (broker, namespace, topic) are `false` or `null`.
+
+   If a higher-level policy later evaluates to `true` (e.g., due to topic reload or namespace/broker policy change),
+   the broker will reapply the effective policy and mark the subscription as replicated again.
+
+### Design Consistency with Consumer-level Logic
+
+This design builds upon the existing **consumer-level logic**, which already follows **enable-only semantics**:
+
+```java
+// https://github.com/apache/pulsar/blob/8798a4662300dfa3d330dbb0277073874cc2d86e/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentTopic.java#L1153-L1156
+if (replicated != null && replicated && !subscription.isReplicated()) {
+    subscription.setReplicated(true);
+}
+```
+
+Key points:
+
+* **Consumer-level default changed** from `false` to `null` ([PR #23757](https://github.com/apache/pulsar/pull/23757)).
+  * Combined with the multi-level policy evaluation, this ensures **compatibility across all client languages and versions**:
+      * Older clients defaulting to `false`
+      * Newer clients defaulting to `null`
+  * Both behave correctly with **broker, namespace, and topic-level policies**.
+
+# Detailed Design
+
+## Design & Implementation Details
+
+### Broker level
+
+Add the field `replicate_subscriptions_state` to the `org.apache.pulsar.broker.ServiceConfiguration` class
+to control subscription replication at the broker level:
+
+```java
+public class ServiceConfiguration implements PulsarConfiguration {
+    @FieldContext(
+            category = CATEGORY_SERVER,
+            required = false,
+            dynamic = false,
+            doc = "The default value for replicating subscription state."
+    )
+    private Boolean replicate_subscriptions_state;
+}
+```
+
+### Namespace level
+
+1. Add the field `replicate_subscriptions_state` to the `org.apache.pulsar.common.policies.data.Policies` class
+   to control subscription replication at the namespace level:
+    ```java
+    public class Policies {
+        @SuppressWarnings("checkstyle:MemberName")
+        public Boolean replicate_subscription_state;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.Namespaces` interface:
+     ```java
+     public interface Namespaces {
+         void setReplicateSubscriptionState(String namespace, boolean enabled) throws PulsarAdminException;
+         CompletableFuture<Void> setReplicateSubscriptionStateAsync(String namespace, boolean enabled);
+         Boolean getReplicateSubscriptionState(String namespace) throws PulsarAdminException;
+         CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String namespace);
+         void removeReplicateSubscriptionState(String namespace) throws PulsarAdminException;
+         CompletableFuture<Void> removeReplicateSubscriptionStateAsync(String namespace);
+     }
+     ```
+3. Implement the management methods in the `org.apache.pulsar.client.admin.internal.NamespacesImpl` class.
+
+### Topic level
+
+1. Add the field `Boolean replicateSubscriptionState` to the `org.apache.pulsar.common.policies.data.TopicPolicies`
+   class to enable subscription replication at the topic level:
+   ```java
+    public class TopicPolicies {
+         public Boolean replicateSubscriptionState;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.TopicPolicies` interface:
+   ```java
+       public interface TopicPolicies {
+           void setReplicateSubscriptionState(String topic, boolean enabled) throws PulsarAdminException;
+           CompletableFuture<Void> setReplicateSubscriptionStateAsync(String topic, boolean enabled);
+           Boolean getReplicateSubscriptionState(String topic, boolean applied) throws PulsarAdminException;
+           CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String topic, boolean applied);
+           void removeReplicateSubscriptionState(String topic) throws PulsarAdminException;
+           CompletableFuture<Void> removeReplicateSubscriptionStateAsync(String topic);
+       }
+   ```
+3. Implement the management methods in the `org.apache.pulsar.client.admin.internal.TopicPoliciesImpl` class.
+
+### Consumer level
+
+No changes. When the consumer with `replicateSubscriptionState=true`, the subscription will be snapshot. If `false`, no
+operation will be performed.
+
+## Public-facing Changes
+
+### Public API
+
+> **Note:** Setting `false` at namespace or topic level is accepted but not applied (enable-only semantics). Use the
+`DELETE` endpoint or `remove-` command to clear the configuration instead.
+
+##### Namespace level
+
+- `POST /{tenant}/{namespace}/replicateSubscriptionState`: set the subscription replication configuration on the
+  namespace level.
+    - Content-Type: `application/json`
+    - Body: `true` or `false`.
+- `GET /{tenant}/{namespace}/replicateSubscriptionState`: get subscription replication configuration on the namespace
+  level.
+    - Response: `true`, `false`, or `null` (not configured)
+- `DELETE /{tenant}/{namespace}/replicateSubscriptionState`: remove the subscription replication configuration on the
+  namespace level.
+
+##### Topic level
+
+- `POST /{tenant}/{namespace}/{topic}/replicateSubscriptionState`: set the subscription replication configuration on
+  the topic level.
+    - Content-Type: `application/json`
+    - Body: `true` or `false`.
+- `GET /{tenant}/{namespace}/{topic}/replicateSubscriptionState`: get subscription replication configuration on the
+  topic level.
+    - Parameters:
+        - `applied=true`: get the effective configuration following `Topic > Namespace > Broker` precedence.
+        - `applied=false`: get the topic level configuration only.
+    - Response: `true`, `false`, or `null` (not configured)
+- `DELETE /{tenant}/{namespace}/{topic}/replicateSubscriptionState`: remove the subscription replication configuration
+  on the topic level.
+
+### CLI
+
+#### Namespace level
+
+- `pulsar-admin namespaces set-replicate-subscription-state <tenant>/<namespace> --enabled true/false` to
+  set the subscription replication on the namespace level.
+- `pulsar-admin namespaces get-replicate-subscription-state <tenant>/<namespace>` to get the subscription
+  replication configuration on the namespace level.
+- `pulsar-admin namespaces remove-replicate-subscription-state <tenant>/<namespace>` to remove the subscription
+  replication configuration on the namespace level.
+
+#### Topic level
+
+- `pulsar-admin topicPolicies set-replicate-subscription-state <tenant>/<namespace>/<topic> --enabled true/false`
+  to set the subscription replication on the topic level.
+- `pulsar-admin topicPolicies get-replicate-subscription-state <tenant>/<namespace>/<topic>` to get the
+  subscription replication configuration on the topic level.
+- `pulsar-admin topicPolicies remove-replicate-subscription-state <tenant>/<namespace>/<topic>` to remove the
+  subscription replication configuration on the topic level.
+
+# Security Considerations
+
+Both write and read operations require the necessary permissions, which already exist in Pulsar.
+
+## Namespace level
+
+- Write the subscription replication configuration:
+    - Required: `PolicyName.REPLICATED_SUBSCRIPTION` with `WRITE` permission.
+- Read the subscription replication configuration:
+    - Required: `PolicyName.REPLICATED_SUBSCRIPTION` with `READ` permission.
+
+## Topic level
+
+- Write the subscription replication configuration
+    - Required: `TopicOperation.SET_REPLICATED_SUBSCRIPTION_STATUS` permission.
+- Read the subscription replication configuration:
+    - Required: `TopicOperation.GET_REPLICATED_SUBSCRIPTION_STATUS` permission.
+
+# Backward & Forward Compatibility
+
+## Upgrade
+
+None.
+
+## Downgrade / Rollback
+
+If the broker is downgrade, users need to ensure that the subscription replication configuration is reset to maintain
+the correct replication behavior by the pulsar-admin CLI or API:

Review Comment:
   Grammar fix: 'If the broker is downgrade' should be 'If the broker is downgraded' (or 'If the broker is downgraded/rolled back').
   ```suggestion
   If the broker is downgraded, users need to ensure that the subscription replication configuration is reset to maintain
   the correct replication behavior via the pulsar-admin CLI or API:
   ```



##########
pip/pip-398.md:
##########
@@ -0,0 +1,291 @@
+# PIP-398: Subscription replication on the broker, namespace and topic levels
+
+# Background knowledge
+
+https://github.com/apache/pulsar/pull/4299 introduces the subscription replication feature on the consumer level:
+
+```java
+Consumer<byte[]> consumer = pulsarClient.newConsumer().topic("topic").replicateSubscriptionState(true /* false */)
+        .subscriptionName("sub").subscribe();
+```
+
+While this provides flexibility, it introduces overhead in managing replication for a large number of consumers. Users
+need to manually enable the `replicateSubscriptionState` flag for each consumer, which can become cumbersome in
+large-scale deployments.
+
+# Motivation
+
+The key motivation behind this PIP is to simplify subscription replication configuration, especially in failover
+scenarios. When a main cluster goes down and a backup cluster is activated, ensuring that subscription states are
+consistently replicated across clusters is critical for failover scenarios. By extending the replication configuration
+to the broker, namespace and topic levels, the system reduces the need for explicit consumer-level configuration.
+
+# Goals
+
+## In Scope
+
+- Provide management of subscription replication at the broker, namespace and topic levels using the Pulsar Admin CLI
+  and API.
+- Define the priority order for subscription replication configuration across different levels.
+
+## Out of Scope
+
+- Changes to the existing consumer-level `replicateSubscriptionState` configuration.
+- Changes to the subscription replication mechanism itself (how states are replicated across clusters).
+
+# High Level Design

Review Comment:
   The document uses multiple top-level `#` headings after the title. For consistent Markdown structure (and better TOC rendering), section headings after `# PIP-398: ...` should typically be `## ...` (and deeper sections `### ...`).



##########
pip/pip-398.md:
##########
@@ -0,0 +1,291 @@
+# PIP-398: Subscription replication on the broker, namespace and topic levels
+
+# Background knowledge
+
+https://github.com/apache/pulsar/pull/4299 introduces the subscription replication feature on the consumer level:
+
+```java
+Consumer<byte[]> consumer = pulsarClient.newConsumer().topic("topic").replicateSubscriptionState(true /* false */)
+        .subscriptionName("sub").subscribe();
+```
+
+While this provides flexibility, it introduces overhead in managing replication for a large number of consumers. Users
+need to manually enable the `replicateSubscriptionState` flag for each consumer, which can become cumbersome in
+large-scale deployments.
+
+# Motivation
+
+The key motivation behind this PIP is to simplify subscription replication configuration, especially in failover
+scenarios. When a main cluster goes down and a backup cluster is activated, ensuring that subscription states are
+consistently replicated across clusters is critical for failover scenarios. By extending the replication configuration
+to the broker, namespace and topic levels, the system reduces the need for explicit consumer-level configuration.
+
+# Goals
+
+## In Scope
+
+- Provide management of subscription replication at the broker, namespace and topic levels using the Pulsar Admin CLI
+  and API.
+- Define the priority order for subscription replication configuration across different levels.
+
+## Out of Scope
+
+- Changes to the existing consumer-level `replicateSubscriptionState` configuration.
+- Changes to the subscription replication mechanism itself (how states are replicated across clusters).
+
+# High Level Design
+
+This PIP introduces the `replicateSubscriptionState` configuration at the **broker, namespace, and topic levels** to
+simplify the management of subscription replication.
+
+With this proposal, administrators can configure subscription replication at higher levels so that consumers
+automatically inherit the effective configuration.
+
+# Enable-only Semantics & Effective Logic
+
+## Enable-only Semantics
+
+Subscription replication policies follow Pulsar's hierarchical configuration model:
+
+```
+Topic > Namespace > Broker
+```
+
+The **highest level with a configured value determines the effective policy**; lower levels are ignored once a
+higher-level configuration exists.
+
+* **Broker, Namespace, and Topic levels** follow **enable-only semantics**:
+    * `true` → enables subscription replication.
+    * `false` or `null` → do **not disable** existing replicated subscriptions.
+
+* **Consumer configuration** participates in the evaluation but is **not part of the policy hierarchy**:
+    * `true` → enables replication.
+    * `false` → does **not automatically disable** replication.
+    * `null` → inherits the effective configuration from higher-level policies.
+
+Explicitly **disabling replication** for a subscription is only possible via the **Pulsar Admin API** targeting that
+subscription.
+
+## Effective Replication Logic
+
+The broker determines the replication state for each subscription as follows:
+
+1. Determine the **effective policy** using Pulsar's precedence: `Topic > Namespace > Broker`.
+
+2. If the **effective policy is `true`**, the subscription is marked as replicated.
+
+3. If the effective policy is not `true`, the broker evaluates the **consumer configuration**:
+    * `true` → enables replication
+    * `false` or `null` → no change to the existing subscription state
+
+4. A subscription can be explicitly disabled via **Pulsar Admin**, but this succeeds **only if all higher-level policies
+   ** (broker, namespace, topic) are `false` or `null`.
+
+   If a higher-level policy later evaluates to `true` (e.g., due to topic reload or namespace/broker policy change),
+   the broker will reapply the effective policy and mark the subscription as replicated again.
+
+### Design Consistency with Consumer-level Logic
+
+This design builds upon the existing **consumer-level logic**, which already follows **enable-only semantics**:
+
+```java
+// https://github.com/apache/pulsar/blob/8798a4662300dfa3d330dbb0277073874cc2d86e/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentTopic.java#L1153-L1156
+if (replicated != null && replicated && !subscription.isReplicated()) {
+    subscription.setReplicated(true);
+}
+```
+
+Key points:
+
+* **Consumer-level default changed** from `false` to `null` ([PR #23757](https://github.com/apache/pulsar/pull/23757)).
+  * Combined with the multi-level policy evaluation, this ensures **compatibility across all client languages and versions**:
+      * Older clients defaulting to `false`
+      * Newer clients defaulting to `null`
+  * Both behave correctly with **broker, namespace, and topic-level policies**.
+
+# Detailed Design
+
+## Design & Implementation Details
+
+### Broker level
+
+Add the field `replicate_subscriptions_state` to the `org.apache.pulsar.broker.ServiceConfiguration` class
+to control subscription replication at the broker level:
+
+```java
+public class ServiceConfiguration implements PulsarConfiguration {
+    @FieldContext(
+            category = CATEGORY_SERVER,
+            required = false,
+            dynamic = false,
+            doc = "The default value for replicating subscription state."
+    )
+    private Boolean replicate_subscriptions_state;

Review Comment:
   The proposed config/property name is inconsistent: the text says `replicate_subscriptions_state` (plural `subscriptions`) while the `Policies` snippet uses `replicate_subscription_state` (singular `subscription`). This will be confusing for implementers and API consumers; align the name consistently across broker/namespace/topic representations (and ensure the chosen name matches the intended config key/public API).



##########
pip/pip-398.md:
##########
@@ -0,0 +1,291 @@
+# PIP-398: Subscription replication on the broker, namespace and topic levels
+
+# Background knowledge

Review Comment:
   The document uses multiple top-level `#` headings after the title. For consistent Markdown structure (and better TOC rendering), section headings after `# PIP-398: ...` should typically be `## ...` (and deeper sections `### ...`).



##########
pip/pip-398.md:
##########
@@ -0,0 +1,291 @@
+# PIP-398: Subscription replication on the broker, namespace and topic levels
+
+# Background knowledge
+
+https://github.com/apache/pulsar/pull/4299 introduces the subscription replication feature on the consumer level:
+
+```java
+Consumer<byte[]> consumer = pulsarClient.newConsumer().topic("topic").replicateSubscriptionState(true /* false */)
+        .subscriptionName("sub").subscribe();
+```
+
+While this provides flexibility, it introduces overhead in managing replication for a large number of consumers. Users
+need to manually enable the `replicateSubscriptionState` flag for each consumer, which can become cumbersome in
+large-scale deployments.
+
+# Motivation
+
+The key motivation behind this PIP is to simplify subscription replication configuration, especially in failover
+scenarios. When a main cluster goes down and a backup cluster is activated, ensuring that subscription states are
+consistently replicated across clusters is critical for failover scenarios. By extending the replication configuration
+to the broker, namespace and topic levels, the system reduces the need for explicit consumer-level configuration.
+
+# Goals
+
+## In Scope
+
+- Provide management of subscription replication at the broker, namespace and topic levels using the Pulsar Admin CLI
+  and API.
+- Define the priority order for subscription replication configuration across different levels.
+
+## Out of Scope
+
+- Changes to the existing consumer-level `replicateSubscriptionState` configuration.
+- Changes to the subscription replication mechanism itself (how states are replicated across clusters).
+
+# High Level Design
+
+This PIP introduces the `replicateSubscriptionState` configuration at the **broker, namespace, and topic levels** to
+simplify the management of subscription replication.
+
+With this proposal, administrators can configure subscription replication at higher levels so that consumers
+automatically inherit the effective configuration.
+
+# Enable-only Semantics & Effective Logic
+
+## Enable-only Semantics
+
+Subscription replication policies follow Pulsar's hierarchical configuration model:
+
+```
+Topic > Namespace > Broker
+```
+
+The **highest level with a configured value determines the effective policy**; lower levels are ignored once a
+higher-level configuration exists.
+
+* **Broker, Namespace, and Topic levels** follow **enable-only semantics**:
+    * `true` → enables subscription replication.
+    * `false` or `null` → do **not disable** existing replicated subscriptions.
+
+* **Consumer configuration** participates in the evaluation but is **not part of the policy hierarchy**:
+    * `true` → enables replication.
+    * `false` → does **not automatically disable** replication.
+    * `null` → inherits the effective configuration from higher-level policies.
+
+Explicitly **disabling replication** for a subscription is only possible via the **Pulsar Admin API** targeting that
+subscription.
+
+## Effective Replication Logic
+
+The broker determines the replication state for each subscription as follows:
+
+1. Determine the **effective policy** using Pulsar's precedence: `Topic > Namespace > Broker`.
+
+2. If the **effective policy is `true`**, the subscription is marked as replicated.
+
+3. If the effective policy is not `true`, the broker evaluates the **consumer configuration**:
+    * `true` → enables replication
+    * `false` or `null` → no change to the existing subscription state
+
+4. A subscription can be explicitly disabled via **Pulsar Admin**, but this succeeds **only if all higher-level policies
+   ** (broker, namespace, topic) are `false` or `null`.
+
+   If a higher-level policy later evaluates to `true` (e.g., due to topic reload or namespace/broker policy change),
+   the broker will reapply the effective policy and mark the subscription as replicated again.
+
+### Design Consistency with Consumer-level Logic
+
+This design builds upon the existing **consumer-level logic**, which already follows **enable-only semantics**:
+
+```java
+// https://github.com/apache/pulsar/blob/8798a4662300dfa3d330dbb0277073874cc2d86e/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentTopic.java#L1153-L1156
+if (replicated != null && replicated && !subscription.isReplicated()) {
+    subscription.setReplicated(true);
+}
+```
+
+Key points:
+
+* **Consumer-level default changed** from `false` to `null` ([PR #23757](https://github.com/apache/pulsar/pull/23757)).
+  * Combined with the multi-level policy evaluation, this ensures **compatibility across all client languages and versions**:
+      * Older clients defaulting to `false`
+      * Newer clients defaulting to `null`
+  * Both behave correctly with **broker, namespace, and topic-level policies**.
+
+# Detailed Design
+
+## Design & Implementation Details
+
+### Broker level
+
+Add the field `replicate_subscriptions_state` to the `org.apache.pulsar.broker.ServiceConfiguration` class

Review Comment:
   The proposed config/property name is inconsistent: the text says `replicate_subscriptions_state` (plural `subscriptions`) while the `Policies` snippet uses `replicate_subscription_state` (singular `subscription`). This will be confusing for implementers and API consumers; align the name consistently across broker/namespace/topic representations (and ensure the chosen name matches the intended config key/public API).



##########
pip/pip-398.md:
##########
@@ -0,0 +1,291 @@
+# PIP-398: Subscription replication on the broker, namespace and topic levels
+
+# Background knowledge
+
+https://github.com/apache/pulsar/pull/4299 introduces the subscription replication feature on the consumer level:
+
+```java
+Consumer<byte[]> consumer = pulsarClient.newConsumer().topic("topic").replicateSubscriptionState(true /* false */)
+        .subscriptionName("sub").subscribe();
+```
+
+While this provides flexibility, it introduces overhead in managing replication for a large number of consumers. Users
+need to manually enable the `replicateSubscriptionState` flag for each consumer, which can become cumbersome in
+large-scale deployments.
+
+# Motivation
+
+The key motivation behind this PIP is to simplify subscription replication configuration, especially in failover
+scenarios. When a main cluster goes down and a backup cluster is activated, ensuring that subscription states are
+consistently replicated across clusters is critical for failover scenarios. By extending the replication configuration
+to the broker, namespace and topic levels, the system reduces the need for explicit consumer-level configuration.
+
+# Goals
+
+## In Scope
+
+- Provide management of subscription replication at the broker, namespace and topic levels using the Pulsar Admin CLI
+  and API.
+- Define the priority order for subscription replication configuration across different levels.
+
+## Out of Scope
+
+- Changes to the existing consumer-level `replicateSubscriptionState` configuration.
+- Changes to the subscription replication mechanism itself (how states are replicated across clusters).
+
+# High Level Design
+
+This PIP introduces the `replicateSubscriptionState` configuration at the **broker, namespace, and topic levels** to
+simplify the management of subscription replication.
+
+With this proposal, administrators can configure subscription replication at higher levels so that consumers
+automatically inherit the effective configuration.
+
+# Enable-only Semantics & Effective Logic
+
+## Enable-only Semantics
+
+Subscription replication policies follow Pulsar's hierarchical configuration model:
+
+```
+Topic > Namespace > Broker
+```
+
+The **highest level with a configured value determines the effective policy**; lower levels are ignored once a
+higher-level configuration exists.
+
+* **Broker, Namespace, and Topic levels** follow **enable-only semantics**:
+    * `true` → enables subscription replication.
+    * `false` or `null` → do **not disable** existing replicated subscriptions.
+
+* **Consumer configuration** participates in the evaluation but is **not part of the policy hierarchy**:
+    * `true` → enables replication.
+    * `false` → does **not automatically disable** replication.

Review Comment:
   There’s a semantic inconsistency: earlier text implies `false` is a configured value (which would participate in precedence and potentially override lower levels), but later the note says `false` is 'accepted but not applied' (suggesting it behaves like 'unset'). Please clarify whether `false` is stored and affects precedence (even if it doesn’t actively disable existing subscriptions), or whether it should be treated as equivalent to `null`/not-configured and therefore should not override a lower-level `true`.
   ```suggestion
   The **highest level that is explicitly configured to `true` determines the effective policy**; lower levels are ignored
   once such a higher-level configuration exists.
   
   * **Broker, Namespace, and Topic levels** follow **enable-only semantics**:
       * `true` → enables subscription replication and is treated as a configured value for precedence.
       * `false` or `null` → are treated as **not configured**: they do **not disable** existing replicated subscriptions
         and do **not override** a lower-level `true`.
   
   * **Consumer configuration** participates in the evaluation but is **not part of the policy hierarchy**:
       * `true` → enables replication.
       * `false` → expresses that the consumer does not request replication but does **not automatically disable**
         replication if enabled by higher-level policies.
   ```



##########
pip/pip-398.md:
##########
@@ -0,0 +1,291 @@
+# PIP-398: Subscription replication on the broker, namespace and topic levels
+
+# Background knowledge
+
+https://github.com/apache/pulsar/pull/4299 introduces the subscription replication feature on the consumer level:
+
+```java
+Consumer<byte[]> consumer = pulsarClient.newConsumer().topic("topic").replicateSubscriptionState(true /* false */)
+        .subscriptionName("sub").subscribe();
+```
+
+While this provides flexibility, it introduces overhead in managing replication for a large number of consumers. Users
+need to manually enable the `replicateSubscriptionState` flag for each consumer, which can become cumbersome in
+large-scale deployments.
+
+# Motivation
+
+The key motivation behind this PIP is to simplify subscription replication configuration, especially in failover
+scenarios. When a main cluster goes down and a backup cluster is activated, ensuring that subscription states are
+consistently replicated across clusters is critical for failover scenarios. By extending the replication configuration
+to the broker, namespace and topic levels, the system reduces the need for explicit consumer-level configuration.
+
+# Goals
+
+## In Scope
+
+- Provide management of subscription replication at the broker, namespace and topic levels using the Pulsar Admin CLI
+  and API.
+- Define the priority order for subscription replication configuration across different levels.
+
+## Out of Scope
+
+- Changes to the existing consumer-level `replicateSubscriptionState` configuration.
+- Changes to the subscription replication mechanism itself (how states are replicated across clusters).
+
+# High Level Design
+
+This PIP introduces the `replicateSubscriptionState` configuration at the **broker, namespace, and topic levels** to
+simplify the management of subscription replication.
+
+With this proposal, administrators can configure subscription replication at higher levels so that consumers
+automatically inherit the effective configuration.
+
+# Enable-only Semantics & Effective Logic
+
+## Enable-only Semantics
+
+Subscription replication policies follow Pulsar's hierarchical configuration model:
+
+```
+Topic > Namespace > Broker
+```
+
+The **highest level with a configured value determines the effective policy**; lower levels are ignored once a
+higher-level configuration exists.
+
+* **Broker, Namespace, and Topic levels** follow **enable-only semantics**:
+    * `true` → enables subscription replication.
+    * `false` or `null` → do **not disable** existing replicated subscriptions.
+
+* **Consumer configuration** participates in the evaluation but is **not part of the policy hierarchy**:
+    * `true` → enables replication.
+    * `false` → does **not automatically disable** replication.
+    * `null` → inherits the effective configuration from higher-level policies.
+
+Explicitly **disabling replication** for a subscription is only possible via the **Pulsar Admin API** targeting that
+subscription.
+
+## Effective Replication Logic
+
+The broker determines the replication state for each subscription as follows:
+
+1. Determine the **effective policy** using Pulsar's precedence: `Topic > Namespace > Broker`.
+
+2. If the **effective policy is `true`**, the subscription is marked as replicated.
+
+3. If the effective policy is not `true`, the broker evaluates the **consumer configuration**:
+    * `true` → enables replication
+    * `false` or `null` → no change to the existing subscription state
+
+4. A subscription can be explicitly disabled via **Pulsar Admin**, but this succeeds **only if all higher-level policies
+   ** (broker, namespace, topic) are `false` or `null`.
+
+   If a higher-level policy later evaluates to `true` (e.g., due to topic reload or namespace/broker policy change),
+   the broker will reapply the effective policy and mark the subscription as replicated again.
+
+### Design Consistency with Consumer-level Logic
+
+This design builds upon the existing **consumer-level logic**, which already follows **enable-only semantics**:
+
+```java
+// https://github.com/apache/pulsar/blob/8798a4662300dfa3d330dbb0277073874cc2d86e/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentTopic.java#L1153-L1156
+if (replicated != null && replicated && !subscription.isReplicated()) {
+    subscription.setReplicated(true);
+}
+```
+
+Key points:
+
+* **Consumer-level default changed** from `false` to `null` ([PR #23757](https://github.com/apache/pulsar/pull/23757)).
+  * Combined with the multi-level policy evaluation, this ensures **compatibility across all client languages and versions**:
+      * Older clients defaulting to `false`
+      * Newer clients defaulting to `null`
+  * Both behave correctly with **broker, namespace, and topic-level policies**.
+
+# Detailed Design
+
+## Design & Implementation Details
+
+### Broker level
+
+Add the field `replicate_subscriptions_state` to the `org.apache.pulsar.broker.ServiceConfiguration` class
+to control subscription replication at the broker level:
+
+```java
+public class ServiceConfiguration implements PulsarConfiguration {
+    @FieldContext(
+            category = CATEGORY_SERVER,
+            required = false,
+            dynamic = false,
+            doc = "The default value for replicating subscription state."
+    )
+    private Boolean replicate_subscriptions_state;
+}
+```
+
+### Namespace level
+
+1. Add the field `replicate_subscriptions_state` to the `org.apache.pulsar.common.policies.data.Policies` class
+   to control subscription replication at the namespace level:
+    ```java
+    public class Policies {
+        @SuppressWarnings("checkstyle:MemberName")
+        public Boolean replicate_subscription_state;

Review Comment:
   The proposed config/property name is inconsistent: the text says `replicate_subscriptions_state` (plural `subscriptions`) while the `Policies` snippet uses `replicate_subscription_state` (singular `subscription`). This will be confusing for implementers and API consumers; align the name consistently across broker/namespace/topic representations (and ensure the chosen name matches the intended config key/public API).
   ```suggestion
           public Boolean replicate_subscriptions_state;
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the broker, namespace and topic levels [pulsar]

Posted by "nodece (via GitHub)" <gi...@apache.org>.
nodece commented on code in PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#discussion_r2909764921


##########
pip/pip-398.md:
##########
@@ -0,0 +1,253 @@
+# PIP-398: Subscription replication on the broker, namespace and topic levels
+
+# Background knowledge
+
+https://github.com/apache/pulsar/pull/4299 introduces the subscription replication feature on the consumer level:
+
+```java
+Consumer<byte[]> consumer = pulsarClient.newConsumer().topic("topic").replicateSubscriptionState(false/*true*/)
+        .subscriptionName("sub").subscribe();
+```
+
+While this provides flexibility, it introduces overhead in managing replication for a large number of consumers. Users
+need to manually enable the `replicateSubscriptionState` flag for each consumer, which can become cumbersome in
+large-scale deployments.
+
+# Motivation
+
+The key motivation behind this PIP is to simplify subscription replication configuration, especially in failover
+scenarios. When a main cluster goes down and a backup cluster is activated, ensuring that subscription states are
+consistently replicated across clusters is critical for failover scenarios. By extending the replication configuration
+to the broker, namespace and topic levels, the system reduces the need for explicit consumer-level configuration.
+
+# Goals
+
+## In Scope
+
+The PIP aims to provide management of subscription replication at the broker, namespace and topic levels using the
+Pulsar Admin CLI and API.
+
+# High Level Design
+
+Introduces the `replicateSubscriptionState` configuration to enabling subscription replication on the broker, namespace
+and topic levels, when enabled, all consumers under the broker/namespace/topic will automatically replicate their
+subscription states to remote clusters.
+
+The priority for the subscription replication configuration is as follows:
+
+- consumer level > topic level > namespace level > broker level.
+- If `replicateSubscriptionState` is set at the consumer level, configurations at the topic, namespace, and broker levels are
+  ignored.
+- If set at the topic level, the namespace-level configuration is ignored.
+- If set at the namespace level, the broker-level configuration is ignored.

Review Comment:
   @lhotari You are right.
   
   The replicated flag at the subscription level is persisted in BookKeeper cursor metadata, and only `true` indicates a replicated subscription: https://github.com/apache/pulsar/blob/v4.0.9/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentTopic.java#L1149-L1152
   
   To keep the semantics consistent across subscription, topic, and namespace
   levels, the policy only treats `Boolean.TRUE` as enabling the feature. `false` or `null` both mean "not enabling it".
   
   Otherwise, the topic could end up in inconsistent states where a subscription is replicated while the topic policy explicitly disables this.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the broker, namespace and topic levels [pulsar]

Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#issuecomment-2673798407

   > @lhotari Do you have more suggestions? If not, I will send a vote to the mailing list.
   
   @nodece I think that at the topic and namespace level, it could be better to combine `overwriteConsumerReplicateSubscriptionState` (you mentioned it in https://github.com/apache/pulsar/pull/23770#issuecomment-2560597473) and `replicateSubscriptionState` into a single enum instead of 2 separate boolean values to be used in policies. Let's say `ReplicateSubscriptionStatePolicy` enum with values:
   * `OVERRIDE_ENABLE` - overrides any value set by the consumer (or lower level policy) and enables subscription replication
   * `OVERRIDE_DISABLE` - overrides any value set by the consumer (or lower level policy) and enables subscription replication
   * `ENABLE` - enables subscription replication if not explicitly disabled by the consumer (or lower level policy)
   * `DISABLE` - disables subscription replication if not explicitly enabled by the consumer (or lower level policy)
   I don't know if this is accurate, but putting more thought in how to address the topic and namespace level would be useful. 
   
   I haven't seen the APIs being updated to have this information in the policies. I don't think that introducing a completely separate concept for topic and namespace level makes sense.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the broker, namespace and topic levels [pulsar]

Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on code in PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#discussion_r2304792666


##########
pip/pip-398.md:
##########
@@ -0,0 +1,253 @@
+# PIP-398: Subscription replication on the broker, namespace and topic levels
+
+# Background knowledge
+
+https://github.com/apache/pulsar/pull/4299 introduces the subscription replication feature on the consumer level:
+
+```java
+Consumer<byte[]> consumer = pulsarClient.newConsumer().topic("topic").replicateSubscriptionState(false/*true*/)
+        .subscriptionName("sub").subscribe();
+```
+
+While this provides flexibility, it introduces overhead in managing replication for a large number of consumers. Users
+need to manually enable the `replicateSubscriptionState` flag for each consumer, which can become cumbersome in
+large-scale deployments.
+
+# Motivation
+
+The key motivation behind this PIP is to simplify subscription replication configuration, especially in failover
+scenarios. When a main cluster goes down and a backup cluster is activated, ensuring that subscription states are
+consistently replicated across clusters is critical for failover scenarios. By extending the replication configuration
+to the broker, namespace and topic levels, the system reduces the need for explicit consumer-level configuration.
+
+# Goals
+
+## In Scope
+
+The PIP aims to provide management of subscription replication at the broker, namespace and topic levels using the
+Pulsar Admin CLI and API.
+
+# High Level Design
+
+Introduces the `replicateSubscriptionState` configuration to enabling subscription replication on the broker, namespace
+and topic levels, when enabled, all consumers under the broker/namespace/topic will automatically replicate their
+subscription states to remote clusters.
+
+The priority for the subscription replication configuration is as follows:
+
+- consumer level > topic level > namespace level > broker level.
+- If `replicateSubscriptionState` is set at the consumer level, configurations at the topic, namespace, and broker levels are
+  ignored.
+- If set at the topic level, the namespace-level configuration is ignored.
+- If set at the namespace level, the broker-level configuration is ignored.
+
+# Detailed Design
+
+## Design & Implementation Details
+
+### Broker level
+
+Add the field `Boolean replicate_subscriptions_state` to the `org.apache.pulsar.broker.ServiceConfiguration` class
+to control subscription replication at the broker level:
+```java
+public class ServiceConfiguration implements PulsarConfiguration {
+    @FieldContext(
+            category = CATEGORY_SERVER,
+            required = false,
+            dynamic = true,
+            doc = "The default value for replicating subscription state."
+    )
+    private Boolean replicate_subscriptions_state;
+}
+```
+
+### Namespace level
+
+1. Add the field `Boolean replicate_subscriptions_state` to the `org.apache.pulsar.common.policies.data.Policies` class
+   to control subscription replication at the namespace level:
+    ```java
+    public class Policies {
+        @SuppressWarnings("checkstyle:MemberName")
+        public Boolean replicate_subscription_state;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.Namespaces` interface:
+     ```java
+     public interface Namespaces {
+         void setReplicateSubscriptionState(String namespace, Boolean enabled) throws PulsarAdminException;
+         CompletableFuture<Void> setReplicateSubscriptionStateAsync(String namespace, Boolean enabled);
+         Boolean getReplicateSubscriptionState(String namespace) throws PulsarAdminException;
+         CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String namespace);
+     }
+     ```
+3. Implement the management methods in the `org.apache.pulsar.client.admin.internal.NamespacesImpl` class.
+
+### Topic level
+
+1. Add the field `Boolean replicateSubscriptionState` to the `org.apache.pulsar.common.policies.data.TopicPolicies`
+   class to enable subscription replication at the topic level:
+   ```java
+    public class TopicPolicies {
+         public Boolean replicateSubscriptionState;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.TopicPolicies` interface:
+   ```java
+       public interface TopicPolicies { 
+           void setReplicateSubscriptionState(String topic, Boolean enabled) throws PulsarAdminException;
+           CompletableFuture<Void> setReplicateSubscriptionStateAsync(String topic, Boolean enabled);
+           Boolean getReplicateSubscriptionState(String topic, boolean applied) throws PulsarAdminException;
+           CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String topic, boolean applied);
+       }

Review Comment:
   > We have a `pulsar-admin topics set-replicated-subscription-status` command, the current implementation uses Boolean for similar that, so I followed the same design for consistency and to avoid introducing a new pattern in the CLI/API.
   
   This is a valid argument if we are able to achieve all required use cases with a `Boolean`. 
   
   After thinking about it again, it seems that these choices need to be covered:
   
   1. override any setting by the consumer (or lower level policy) and enable subscription replication for all subscriptions
   2. override any setting by the consumer (or lower level policy) and disable subscription replication for all subscriptions
   3. enable subscription replication if not explicitly disabled by the consumer (or lower level policy) for all subscriptions
   4. no setting at all (higher level setting would take priority)
   
   @nodece With a single `Boolean` value, how would you distinguish between the 2 different cases (1. and 3. above) where subscription replication is enabled?
   
   (with "lower level policy", it means the hierarchy of broker -> namespace -> topic)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the broker, namespace and topic levels [pulsar]

Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#issuecomment-3219466953

   @nodece Do you have plans to take this PIP forward?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the namespace and topic levels [pulsar]

Posted by "nodece (via GitHub)" <gi...@apache.org>.
nodece commented on PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#issuecomment-2560597473

   @yyj8 
   > The default configuration for the client is `replicateSubscriptionState=false`.
   
   The latest client defaults to `null`, please see https://github.com/apache/pulsar/pull/23757.
   
   > In the scenario of cluster migration, we hope that there is no need for business code modification, and the server will control the replication of the state of all subscriptions in the cluster uniformly. If the configuration priority of the client is the highest, and the client does not display the configuration `replicateSubscriptionState=true`, this goes against our original intention of not requiring business code modification.
   My suggestion is to configure the cluster dimension. If `repliceAllSubscriptState=true` is enabled, it will be a mandatory configuration to overwrite the client. If `repliceAllSubscriptState=false`, then use the client's configuration.
   
   I understand your idea, this can make all subscriptions replicated when `repliceAllSubscriptState=true`, when true, this equals the consumer with `replicateSubscriptionState=true`, and they have the highest.
   
   However, this idea conflicts with PIP-398, which assumes that the consumer-level configuration for `replicateSubscriptionState` is not set, and instead, replication behavior is driven by namespace and topic policies. If the `replicateSubscriptionState` is set,  it means that the specified subscription will be replicated, namespace and topic levels cannot change this behavior. If changed, it will result in a breaking change. This is worth considering. 
   
   In our case, I don't want to configure the `replicateSubscriptionState` on the consumer, but the user has been configured, I cannot change the user code. We have the same case, so I want to ignore the `replicateSubscriptionState`. 
   
   ## My idea
   To combine your case and my case, I suggest introducing a broker configuration for overwriting the consume configuration, and assuming the namespace and topic level are set, the final result will be so like:
   
   - `overwirteConsumerReplicateSubscriptionState=true`:  The subscription will be replicated. 
   - `overwirteConsumerReplicateSubscriptionState=false`: The subscription will be replicated. 
   - `overwirteConsumerReplicateSubscriptionState=null`: This mean the consumer level is null, and then the broker depends on the namespace and topic level to check the subscription replication.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the namespace and topic levels [pulsar]

Posted by "yyj8 (via GitHub)" <gi...@apache.org>.
yyj8 commented on PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#issuecomment-2563447508

   > @yyj8
   > 
   > > The default configuration for the client is `replicateSubscriptionState=false`.
   > 
   > The latest client defaults to `null`, please see #23757.
   > 
   > > In the scenario of cluster migration, we hope that there is no need for business code modification, and the server will control the replication of the state of all subscriptions in the cluster uniformly. If the configuration priority of the client is the highest, and the client does not display the configuration `replicateSubscriptionState=true`, this goes against our original intention of not requiring business code modification.
   > > My suggestion is to configure the cluster dimension. If `repliceAllSubscriptState=true` is enabled, it will be a mandatory configuration to overwrite the client. If `repliceAllSubscriptState=false`, then use the client's configuration.
   > 
   > I understand your idea, this can make all subscriptions replicated when `repliceAllSubscriptState=true`, when true, this equals the consumer with `replicateSubscriptionState=true`, and they have the highest.
   > 
   > However, this idea conflicts with PIP-398, which assumes that the consumer-level configuration for `replicateSubscriptionState` is not set, and instead, replication behavior is driven by namespace and topic policies. If the `replicateSubscriptionState` is set, it means that the specified subscription will be replicated, namespace and topic levels cannot change this behavior. If changed, it will result in a breaking change. This is worth considering.
   > 
   > In our case, I don't want to configure the `replicateSubscriptionState` on the consumer, but the user has been configured, I cannot change the user code. We have the same case, so I also want to ignore the `replicateSubscriptionState`.
   > 
   > ## My idea
   > To combine your case and my case, I suggest introducing a broker configuration for overwriting the consume configuration, and assuming the namespace and topic level are set, the final result will be so like this:
   > 
   > * `overwirteConsumerReplicateSubscriptionState=true`:  The subscription will be replicated.
   > * `overwirteConsumerReplicateSubscriptionState=false`: The subscription will be replicated.
   > * `overwirteConsumerReplicateSubscriptionState=null`: This means the consumer level is null, and then the broker depends on the namespace and topic level to check the subscription replication.
   > 
   > We can also introduce `replicateSubscriptionsState=true/false/empty` configuration on the broker level, when the namespace/topic/consumer levels are not set, we use the broker level configuration to check if replicates the subscription.
   
   @nodece I agree with your suggestion.
   Before using cluster level configuration, we first check if there are topic and namespace dimensions. If there are none, we apply cluster level configuration; The priority of the three dimensions from high to low is: topic -> namespace -> cluster.
   
   Then we can make unified code adjustments on your pip, or you can directly merge my code into your code, or after you submit the code, I can synchronize your code and modify the cluster level configuration before submitting.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the broker, namespace and topic levels [pulsar]

Posted by "merlimat (via GitHub)" <gi...@apache.org>.
merlimat commented on PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#issuecomment-6179364069

   Thanks for this proposal. A PIP is accepted through a vote on dev@pulsar.apache.org, and this one had a vote thread there that didn't reach the three binding +1 votes needed. It has been open since December 2024. As part of the open-PR cleanup discussed on dev@, I'm closing PIP proposals that are more than six months old without a passed vote. If you'd like to move it forward, please reopen this PR and start a [DISCUSS] or [VOTE] thread on dev@.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the broker, namespace and topic levels [pulsar]

Posted by "nodece (via GitHub)" <gi...@apache.org>.
nodece commented on code in PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#discussion_r1959571734


##########
pip/pip-398.md:
##########
@@ -0,0 +1,245 @@
+# PIP-398: Subscription replication on the broker, namespace and topic levels
+
+# Background knowledge
+
+https://github.com/apache/pulsar/pull/4299 introduces the subscription replication feature on the consumer level:
+
+```java
+Consumer<byte[]> consumer = pulsarClient.newConsumer().topic("topic").replicateSubscriptionState(false/*true*/)
+        .subscriptionName("sub").subscribe();
+```
+
+While this provides flexibility, it introduces overhead in managing replication for a large number of consumers. Users
+need to manually enable the `replicateSubscriptionState` flag for each consumer, which can become cumbersome in
+large-scale deployments.
+
+# Motivation
+
+The key motivation behind this PIP is to simplify subscription replication configuration, especially in failover
+scenarios. When a main cluster goes down and a backup cluster is activated, ensuring that subscription states are
+consistently replicated across clusters is critical for failover scenarios. By extending the replication configuration
+to the broker, namespace and topic levels, the system reduces the need for explicit consumer-level configuration.
+
+# Goals
+
+## In Scope
+
+The PIP aims to provide management of subscription replication at the broker, namespace and topic levels using the
+Pulsar Admin CLI and API.
+
+# High Level Design
+
+Introduces a configuration used for enabling subscription replication on the broker, namespace and topic levels, when
+enabled, all consumers under the broker/namespace/topic will automatically replicate their subscription states to remote
+clusters.
+
+The priority for the subscription replication configuration is as follows:
+
+- consumer level > topic level > namespace level > broker level.
+- If `replicateSubscriptionState` is set at the consumer level, configurations at the topic, namespace, and broker levels are
+  ignored.
+- If set at the topic level, the namespace-level configuration is ignored.
+- If set at the namespace level, the broker-level configuration is ignored.
+
+# Detailed Design
+
+## Design & Implementation Details
+
+### Broker level
+
+Add the field `Boolean replicate_subscriptions_state` to the `org.apache.pulsar.broker.ServiceConfiguration` class
+to control subscription replication at the broker level:
+```java
+public class ServiceConfiguration implements PulsarConfiguration {
+    @FieldContext(
+            category = CATEGORY_SERVER,
+            required = false,
+            dynamic = true,
+            doc = "The default value for replicating subscription state."
+    )
+    private Boolean replicate_subscriptions_state;
+}
+```
+
+### Namespace level
+
+1. Add the field `Boolean replicate_subscriptions_state` to the `org.apache.pulsar.common.policies.data.Policies` class
+   to control subscription replication at the namespace level:
+    ```java
+    public class Policies {
+        @SuppressWarnings("checkstyle:MemberName")
+        public Boolean replicate_subscription_state;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.Namespaces` interface:
+     ```java
+     public interface Namespaces {
+         void setReplicateSubscriptionState(String namespace, Boolean enabled) throws PulsarAdminException;
+         CompletableFuture<Void> setReplicateSubscriptionStateAsync(String namespace, Boolean enabled);
+         Boolean getReplicateSubscriptionState(String namespace) throws PulsarAdminException;
+         CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String namespace);
+     }
+     ```
+3. Implement the management methods in the `org.apache.pulsar.client.admin.internal.NamespacesImpl` class.
+
+### Topic level
+
+1. Add the field `Boolean replicateSubscriptionState` to the `org.apache.pulsar.common.policies.data.TopicPolicies`
+   class to enable subscription replication at the topic level:
+   ```java
+    public class TopicPolicies {
+         public Boolean replicateSubscriptionState;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.TopicPolicies` interface:
+   ```java
+       public interface TopicPolicies { 
+           void setReplicateSubscriptionState(String topic, Boolean enabled) throws PulsarAdminException;
+           CompletableFuture<Void> setReplicateSubscriptionStateAsync(String topic, Boolean enabled);
+           Boolean getReplicateSubscriptionState(String topic, boolean applied) throws PulsarAdminException;
+           CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String topic, boolean applied);
+       }
+   ```
+3. Implement the management methods in the `org.apache.pulsar.client.admin.internal.TopicPoliciesImpl` class.
+
+### Consumer level
+
+No changes. When the consumer with `replicateSubscriptionState=true`, the old/new subscription will be snapshot. If `false`, no operation will
+be performed.
+
+### Subscription replication applied
+
+When the subscription replication is changed on the broker, namespace or topic level, the subscription replication will
+be applied to all consumers under the broker/namespace/topic.
+
+We exclusively consider the `true` case for replication. If the consumer level is set to `false`, the
+replication configuration defined at the topic, namespace, or broker level will be applied to the subscription.
+
+There is a special case here, if the user intentionally sets `false` at the consumer level, and then set `true` at the
+topic/ns/broker level, this may disrupt the user's behavior. This way, we can minimize changes to the Pulsar public API
+as much as possible.
+
+## Public-facing Changes
+
+### Public API
+
+##### Namespace level
+
+- `/{tenant}/{namespace}/replicateSubscriptionState`: enable/disable/remove the subscription replication on the
+  namespace level.
+    - Method: `POST`
+    - Content-Type: `application/json`
+    - Body:
+        - true
+        - false
+        - null
+- `GET /{tenant}/{namespace}/replicateSubscriptionState` to get subscription replication configuration on the namespace
+  level.
+    - Method: `GET`
+    - Response:
+        - true
+        - false
+        - null

Review Comment:
   Yes.



##########
pip/pip-398.md:
##########
@@ -0,0 +1,245 @@
+# PIP-398: Subscription replication on the broker, namespace and topic levels
+
+# Background knowledge
+
+https://github.com/apache/pulsar/pull/4299 introduces the subscription replication feature on the consumer level:
+
+```java
+Consumer<byte[]> consumer = pulsarClient.newConsumer().topic("topic").replicateSubscriptionState(false/*true*/)
+        .subscriptionName("sub").subscribe();
+```
+
+While this provides flexibility, it introduces overhead in managing replication for a large number of consumers. Users
+need to manually enable the `replicateSubscriptionState` flag for each consumer, which can become cumbersome in
+large-scale deployments.
+
+# Motivation
+
+The key motivation behind this PIP is to simplify subscription replication configuration, especially in failover
+scenarios. When a main cluster goes down and a backup cluster is activated, ensuring that subscription states are
+consistently replicated across clusters is critical for failover scenarios. By extending the replication configuration
+to the broker, namespace and topic levels, the system reduces the need for explicit consumer-level configuration.
+
+# Goals
+
+## In Scope
+
+The PIP aims to provide management of subscription replication at the broker, namespace and topic levels using the
+Pulsar Admin CLI and API.
+
+# High Level Design
+
+Introduces a configuration used for enabling subscription replication on the broker, namespace and topic levels, when
+enabled, all consumers under the broker/namespace/topic will automatically replicate their subscription states to remote
+clusters.
+
+The priority for the subscription replication configuration is as follows:
+
+- consumer level > topic level > namespace level > broker level.
+- If `replicateSubscriptionState` is set at the consumer level, configurations at the topic, namespace, and broker levels are
+  ignored.
+- If set at the topic level, the namespace-level configuration is ignored.
+- If set at the namespace level, the broker-level configuration is ignored.
+
+# Detailed Design
+
+## Design & Implementation Details
+
+### Broker level
+
+Add the field `Boolean replicate_subscriptions_state` to the `org.apache.pulsar.broker.ServiceConfiguration` class
+to control subscription replication at the broker level:
+```java
+public class ServiceConfiguration implements PulsarConfiguration {
+    @FieldContext(
+            category = CATEGORY_SERVER,
+            required = false,
+            dynamic = true,
+            doc = "The default value for replicating subscription state."
+    )
+    private Boolean replicate_subscriptions_state;
+}
+```
+
+### Namespace level
+
+1. Add the field `Boolean replicate_subscriptions_state` to the `org.apache.pulsar.common.policies.data.Policies` class
+   to control subscription replication at the namespace level:
+    ```java
+    public class Policies {
+        @SuppressWarnings("checkstyle:MemberName")
+        public Boolean replicate_subscription_state;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.Namespaces` interface:
+     ```java
+     public interface Namespaces {
+         void setReplicateSubscriptionState(String namespace, Boolean enabled) throws PulsarAdminException;
+         CompletableFuture<Void> setReplicateSubscriptionStateAsync(String namespace, Boolean enabled);
+         Boolean getReplicateSubscriptionState(String namespace) throws PulsarAdminException;
+         CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String namespace);
+     }
+     ```
+3. Implement the management methods in the `org.apache.pulsar.client.admin.internal.NamespacesImpl` class.
+
+### Topic level
+
+1. Add the field `Boolean replicateSubscriptionState` to the `org.apache.pulsar.common.policies.data.TopicPolicies`
+   class to enable subscription replication at the topic level:
+   ```java
+    public class TopicPolicies {
+         public Boolean replicateSubscriptionState;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.TopicPolicies` interface:
+   ```java
+       public interface TopicPolicies { 
+           void setReplicateSubscriptionState(String topic, Boolean enabled) throws PulsarAdminException;
+           CompletableFuture<Void> setReplicateSubscriptionStateAsync(String topic, Boolean enabled);
+           Boolean getReplicateSubscriptionState(String topic, boolean applied) throws PulsarAdminException;
+           CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String topic, boolean applied);
+       }
+   ```
+3. Implement the management methods in the `org.apache.pulsar.client.admin.internal.TopicPoliciesImpl` class.
+
+### Consumer level
+
+No changes. When the consumer with `replicateSubscriptionState=true`, the old/new subscription will be snapshot. If `false`, no operation will
+be performed.
+
+### Subscription replication applied
+
+When the subscription replication is changed on the broker, namespace or topic level, the subscription replication will
+be applied to all consumers under the broker/namespace/topic.
+
+We exclusively consider the `true` case for replication. If the consumer level is set to `false`, the
+replication configuration defined at the topic, namespace, or broker level will be applied to the subscription.
+
+There is a special case here, if the user intentionally sets `false` at the consumer level, and then set `true` at the
+topic/ns/broker level, this may disrupt the user's behavior. This way, we can minimize changes to the Pulsar public API
+as much as possible.
+
+## Public-facing Changes
+
+### Public API
+
+##### Namespace level
+
+- `/{tenant}/{namespace}/replicateSubscriptionState`: enable/disable/remove the subscription replication on the
+  namespace level.
+    - Method: `POST`
+    - Content-Type: `application/json`
+    - Body:
+        - true
+        - false
+        - null
+- `GET /{tenant}/{namespace}/replicateSubscriptionState` to get subscription replication configuration on the namespace
+  level.
+    - Method: `GET`
+    - Response:
+        - true
+        - false
+        - null
+
+##### Topic level
+
+- `/{tenant}/{namespace}/{topic}/replicateSubscriptionState`: enable/disable/remove the subscription replication on the
+  topic level.
+    - Method: `POST`
+    - Content-Type: `application/json`
+    - Body:
+        - true
+        - false
+        - null
+- `/{tenant}/{namespace}/{topic}/replicateSubscriptionState` to get subscription replication configuration on the topic
+  level.
+    - Method: `GET`
+    - Parameters:
+        - `applied=true`: get the applied subscription replication configuration, if topic is not set, return the
+          namespace level configuration.
+        - `applied=false`: get the applied subscription replication configuration, if topic is set, return the topic
+          level configuration.
+    - Response:
+        - true
+        - false
+        - null

Review Comment:
   Yes.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the namespace and topic levels [pulsar]

Posted by "nodece (via GitHub)" <gi...@apache.org>.
nodece commented on PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#issuecomment-2559763518

   @yyj8 
   
   > cluster dimension, namespace dimension, and topic dimension.
   
   Once the consumer level is configured, it is the highest priority, the cluster, namespace, and topic levels will be ignored, please see https://github.com/apache/pulsar/pull/23769#issuecomment-2559221226 for details.
   
   > (2)PR https://github.com/apache/pulsar/pull/23769 add the function of dynamically configuring parameter `replicateAllSubscriptionState` values through the command `bin/pulse admin brokers update-dynamic-config  --config replicateAllSubscriptionState --value true/false`
   
   This is feasible.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the broker, namespace and topic levels [pulsar]

Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on code in PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#discussion_r2905108272


##########
pip/pip-398.md:
##########
@@ -0,0 +1,253 @@
+# PIP-398: Subscription replication on the broker, namespace and topic levels
+
+# Background knowledge
+
+https://github.com/apache/pulsar/pull/4299 introduces the subscription replication feature on the consumer level:
+
+```java
+Consumer<byte[]> consumer = pulsarClient.newConsumer().topic("topic").replicateSubscriptionState(false/*true*/)
+        .subscriptionName("sub").subscribe();
+```
+
+While this provides flexibility, it introduces overhead in managing replication for a large number of consumers. Users
+need to manually enable the `replicateSubscriptionState` flag for each consumer, which can become cumbersome in
+large-scale deployments.
+
+# Motivation
+
+The key motivation behind this PIP is to simplify subscription replication configuration, especially in failover
+scenarios. When a main cluster goes down and a backup cluster is activated, ensuring that subscription states are
+consistently replicated across clusters is critical for failover scenarios. By extending the replication configuration
+to the broker, namespace and topic levels, the system reduces the need for explicit consumer-level configuration.
+
+# Goals
+
+## In Scope
+
+The PIP aims to provide management of subscription replication at the broker, namespace and topic levels using the
+Pulsar Admin CLI and API.
+
+# High Level Design
+
+Introduces the `replicateSubscriptionState` configuration to enabling subscription replication on the broker, namespace
+and topic levels, when enabled, all consumers under the broker/namespace/topic will automatically replicate their
+subscription states to remote clusters.
+
+The priority for the subscription replication configuration is as follows:
+
+- consumer level > topic level > namespace level > broker level.
+- If `replicateSubscriptionState` is set at the consumer level, configurations at the topic, namespace, and broker levels are
+  ignored.
+- If set at the topic level, the namespace-level configuration is ignored.
+- If set at the namespace level, the broker-level configuration is ignored.
+
+# Detailed Design
+
+## Design & Implementation Details
+
+### Broker level
+
+Add the field `Boolean replicate_subscriptions_state` to the `org.apache.pulsar.broker.ServiceConfiguration` class
+to control subscription replication at the broker level:
+```java
+public class ServiceConfiguration implements PulsarConfiguration {
+    @FieldContext(
+            category = CATEGORY_SERVER,
+            required = false,
+            dynamic = true,
+            doc = "The default value for replicating subscription state."
+    )
+    private Boolean replicate_subscriptions_state;
+}
+```
+
+### Namespace level
+
+1. Add the field `Boolean replicate_subscriptions_state` to the `org.apache.pulsar.common.policies.data.Policies` class
+   to control subscription replication at the namespace level:
+    ```java
+    public class Policies {
+        @SuppressWarnings("checkstyle:MemberName")
+        public Boolean replicate_subscription_state;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.Namespaces` interface:
+     ```java
+     public interface Namespaces {
+         void setReplicateSubscriptionState(String namespace, Boolean enabled) throws PulsarAdminException;
+         CompletableFuture<Void> setReplicateSubscriptionStateAsync(String namespace, Boolean enabled);
+         Boolean getReplicateSubscriptionState(String namespace) throws PulsarAdminException;
+         CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String namespace);
+     }
+     ```
+3. Implement the management methods in the `org.apache.pulsar.client.admin.internal.NamespacesImpl` class.
+
+### Topic level
+
+1. Add the field `Boolean replicateSubscriptionState` to the `org.apache.pulsar.common.policies.data.TopicPolicies`
+   class to enable subscription replication at the topic level:
+   ```java
+    public class TopicPolicies {
+         public Boolean replicateSubscriptionState;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.TopicPolicies` interface:
+   ```java
+       public interface TopicPolicies { 
+           void setReplicateSubscriptionState(String topic, Boolean enabled) throws PulsarAdminException;
+           CompletableFuture<Void> setReplicateSubscriptionStateAsync(String topic, Boolean enabled);
+           Boolean getReplicateSubscriptionState(String topic, boolean applied) throws PulsarAdminException;
+           CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String topic, boolean applied);
+       }

Review Comment:
   > Subscription replication is enabled if:
   > 
   > * The consumer sets true
   > * The first non-null policy found in the following order is true: topic → namespace → broker. Once a policy is found (even if false), lower levels are ignored.
   
   Ok. I was thinking about a scenario where there would be a need to ignore and override any setting on the consumer. Even if the consumer would enable replication, it would be possible to turn replication off with policies. With a `Boolean` this wouldn't be possible. Perhaps we don't need to support overriding the client side setting in this case.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the broker, namespace and topic levels [pulsar]

Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#issuecomment-3219731826

   > @lhotari Vote has been started.
   
   @nodece I added a request for changes for the `replicateSubscriptionState` which is very confusing when a `Boolean` is used. It's very hard to understand what meaning `null`, `TRUE` and `FALSE` would have. Using an enum would resolve this confusion since it's possible to document the enum values and make the meaning explicit.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the broker, namespace and topic levels [pulsar]

Posted by "nodece (via GitHub)" <gi...@apache.org>.
nodece commented on code in PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#discussion_r2298445009


##########
pip/pip-398.md:
##########
@@ -0,0 +1,253 @@
+# PIP-398: Subscription replication on the broker, namespace and topic levels
+
+# Background knowledge
+
+https://github.com/apache/pulsar/pull/4299 introduces the subscription replication feature on the consumer level:
+
+```java
+Consumer<byte[]> consumer = pulsarClient.newConsumer().topic("topic").replicateSubscriptionState(false/*true*/)
+        .subscriptionName("sub").subscribe();
+```
+
+While this provides flexibility, it introduces overhead in managing replication for a large number of consumers. Users
+need to manually enable the `replicateSubscriptionState` flag for each consumer, which can become cumbersome in
+large-scale deployments.
+
+# Motivation
+
+The key motivation behind this PIP is to simplify subscription replication configuration, especially in failover
+scenarios. When a main cluster goes down and a backup cluster is activated, ensuring that subscription states are
+consistently replicated across clusters is critical for failover scenarios. By extending the replication configuration
+to the broker, namespace and topic levels, the system reduces the need for explicit consumer-level configuration.
+
+# Goals
+
+## In Scope
+
+The PIP aims to provide management of subscription replication at the broker, namespace and topic levels using the
+Pulsar Admin CLI and API.
+
+# High Level Design
+
+Introduces the `replicateSubscriptionState` configuration to enabling subscription replication on the broker, namespace
+and topic levels, when enabled, all consumers under the broker/namespace/topic will automatically replicate their
+subscription states to remote clusters.
+
+The priority for the subscription replication configuration is as follows:
+
+- consumer level > topic level > namespace level > broker level.
+- If `replicateSubscriptionState` is set at the consumer level, configurations at the topic, namespace, and broker levels are
+  ignored.
+- If set at the topic level, the namespace-level configuration is ignored.
+- If set at the namespace level, the broker-level configuration is ignored.
+
+# Detailed Design
+
+## Design & Implementation Details
+
+### Broker level
+
+Add the field `Boolean replicate_subscriptions_state` to the `org.apache.pulsar.broker.ServiceConfiguration` class
+to control subscription replication at the broker level:
+```java
+public class ServiceConfiguration implements PulsarConfiguration {
+    @FieldContext(
+            category = CATEGORY_SERVER,
+            required = false,
+            dynamic = true,
+            doc = "The default value for replicating subscription state."
+    )
+    private Boolean replicate_subscriptions_state;
+}
+```
+
+### Namespace level
+
+1. Add the field `Boolean replicate_subscriptions_state` to the `org.apache.pulsar.common.policies.data.Policies` class
+   to control subscription replication at the namespace level:
+    ```java
+    public class Policies {
+        @SuppressWarnings("checkstyle:MemberName")
+        public Boolean replicate_subscription_state;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.Namespaces` interface:
+     ```java
+     public interface Namespaces {
+         void setReplicateSubscriptionState(String namespace, Boolean enabled) throws PulsarAdminException;
+         CompletableFuture<Void> setReplicateSubscriptionStateAsync(String namespace, Boolean enabled);
+         Boolean getReplicateSubscriptionState(String namespace) throws PulsarAdminException;
+         CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String namespace);
+     }
+     ```
+3. Implement the management methods in the `org.apache.pulsar.client.admin.internal.NamespacesImpl` class.
+
+### Topic level
+
+1. Add the field `Boolean replicateSubscriptionState` to the `org.apache.pulsar.common.policies.data.TopicPolicies`
+   class to enable subscription replication at the topic level:
+   ```java
+    public class TopicPolicies {
+         public Boolean replicateSubscriptionState;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.TopicPolicies` interface:
+   ```java
+       public interface TopicPolicies { 
+           void setReplicateSubscriptionState(String topic, Boolean enabled) throws PulsarAdminException;
+           CompletableFuture<Void> setReplicateSubscriptionStateAsync(String topic, Boolean enabled);
+           Boolean getReplicateSubscriptionState(String topic, boolean applied) throws PulsarAdminException;
+           CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String topic, boolean applied);
+       }

Review Comment:
   We have a `pulsar-admin topics set-replicated-subscription-status` command, the current implementation uses Boolean for similar that, so I followed the same design for consistency and to avoid introducing a new pattern in the CLI/API.
   
   The meaning is:
   - null → remove configuration(I can add a remove api instead of this)
   - true → enable
   - false → disable (**but true has priority if both are set in the different levels**)
   
   Using an enum would make the meaning explicit, but it introduces additional complexity.



##########
pip/pip-398.md:
##########
@@ -0,0 +1,253 @@
+# PIP-398: Subscription replication on the broker, namespace and topic levels
+
+# Background knowledge
+
+https://github.com/apache/pulsar/pull/4299 introduces the subscription replication feature on the consumer level:
+
+```java
+Consumer<byte[]> consumer = pulsarClient.newConsumer().topic("topic").replicateSubscriptionState(false/*true*/)
+        .subscriptionName("sub").subscribe();
+```
+
+While this provides flexibility, it introduces overhead in managing replication for a large number of consumers. Users
+need to manually enable the `replicateSubscriptionState` flag for each consumer, which can become cumbersome in
+large-scale deployments.
+
+# Motivation
+
+The key motivation behind this PIP is to simplify subscription replication configuration, especially in failover
+scenarios. When a main cluster goes down and a backup cluster is activated, ensuring that subscription states are
+consistently replicated across clusters is critical for failover scenarios. By extending the replication configuration
+to the broker, namespace and topic levels, the system reduces the need for explicit consumer-level configuration.
+
+# Goals
+
+## In Scope
+
+The PIP aims to provide management of subscription replication at the broker, namespace and topic levels using the
+Pulsar Admin CLI and API.
+
+# High Level Design
+
+Introduces the `replicateSubscriptionState` configuration to enabling subscription replication on the broker, namespace
+and topic levels, when enabled, all consumers under the broker/namespace/topic will automatically replicate their
+subscription states to remote clusters.
+
+The priority for the subscription replication configuration is as follows:
+
+- consumer level > topic level > namespace level > broker level.
+- If `replicateSubscriptionState` is set at the consumer level, configurations at the topic, namespace, and broker levels are
+  ignored.
+- If set at the topic level, the namespace-level configuration is ignored.
+- If set at the namespace level, the broker-level configuration is ignored.
+
+# Detailed Design
+
+## Design & Implementation Details
+
+### Broker level
+
+Add the field `Boolean replicate_subscriptions_state` to the `org.apache.pulsar.broker.ServiceConfiguration` class
+to control subscription replication at the broker level:
+```java
+public class ServiceConfiguration implements PulsarConfiguration {
+    @FieldContext(
+            category = CATEGORY_SERVER,
+            required = false,
+            dynamic = true,
+            doc = "The default value for replicating subscription state."
+    )
+    private Boolean replicate_subscriptions_state;
+}
+```
+
+### Namespace level
+
+1. Add the field `Boolean replicate_subscriptions_state` to the `org.apache.pulsar.common.policies.data.Policies` class
+   to control subscription replication at the namespace level:
+    ```java
+    public class Policies {
+        @SuppressWarnings("checkstyle:MemberName")
+        public Boolean replicate_subscription_state;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.Namespaces` interface:
+     ```java
+     public interface Namespaces {
+         void setReplicateSubscriptionState(String namespace, Boolean enabled) throws PulsarAdminException;
+         CompletableFuture<Void> setReplicateSubscriptionStateAsync(String namespace, Boolean enabled);
+         Boolean getReplicateSubscriptionState(String namespace) throws PulsarAdminException;
+         CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String namespace);
+     }
+     ```
+3. Implement the management methods in the `org.apache.pulsar.client.admin.internal.NamespacesImpl` class.
+
+### Topic level
+
+1. Add the field `Boolean replicateSubscriptionState` to the `org.apache.pulsar.common.policies.data.TopicPolicies`
+   class to enable subscription replication at the topic level:
+   ```java
+    public class TopicPolicies {
+         public Boolean replicateSubscriptionState;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.TopicPolicies` interface:
+   ```java
+       public interface TopicPolicies { 
+           void setReplicateSubscriptionState(String topic, Boolean enabled) throws PulsarAdminException;
+           CompletableFuture<Void> setReplicateSubscriptionStateAsync(String topic, Boolean enabled);
+           Boolean getReplicateSubscriptionState(String topic, boolean applied) throws PulsarAdminException;
+           CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String topic, boolean applied);
+       }

Review Comment:
   @lhotari 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the broker, namespace and topic levels [pulsar]

Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on code in PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#discussion_r2304792666


##########
pip/pip-398.md:
##########
@@ -0,0 +1,253 @@
+# PIP-398: Subscription replication on the broker, namespace and topic levels
+
+# Background knowledge
+
+https://github.com/apache/pulsar/pull/4299 introduces the subscription replication feature on the consumer level:
+
+```java
+Consumer<byte[]> consumer = pulsarClient.newConsumer().topic("topic").replicateSubscriptionState(false/*true*/)
+        .subscriptionName("sub").subscribe();
+```
+
+While this provides flexibility, it introduces overhead in managing replication for a large number of consumers. Users
+need to manually enable the `replicateSubscriptionState` flag for each consumer, which can become cumbersome in
+large-scale deployments.
+
+# Motivation
+
+The key motivation behind this PIP is to simplify subscription replication configuration, especially in failover
+scenarios. When a main cluster goes down and a backup cluster is activated, ensuring that subscription states are
+consistently replicated across clusters is critical for failover scenarios. By extending the replication configuration
+to the broker, namespace and topic levels, the system reduces the need for explicit consumer-level configuration.
+
+# Goals
+
+## In Scope
+
+The PIP aims to provide management of subscription replication at the broker, namespace and topic levels using the
+Pulsar Admin CLI and API.
+
+# High Level Design
+
+Introduces the `replicateSubscriptionState` configuration to enabling subscription replication on the broker, namespace
+and topic levels, when enabled, all consumers under the broker/namespace/topic will automatically replicate their
+subscription states to remote clusters.
+
+The priority for the subscription replication configuration is as follows:
+
+- consumer level > topic level > namespace level > broker level.
+- If `replicateSubscriptionState` is set at the consumer level, configurations at the topic, namespace, and broker levels are
+  ignored.
+- If set at the topic level, the namespace-level configuration is ignored.
+- If set at the namespace level, the broker-level configuration is ignored.
+
+# Detailed Design
+
+## Design & Implementation Details
+
+### Broker level
+
+Add the field `Boolean replicate_subscriptions_state` to the `org.apache.pulsar.broker.ServiceConfiguration` class
+to control subscription replication at the broker level:
+```java
+public class ServiceConfiguration implements PulsarConfiguration {
+    @FieldContext(
+            category = CATEGORY_SERVER,
+            required = false,
+            dynamic = true,
+            doc = "The default value for replicating subscription state."
+    )
+    private Boolean replicate_subscriptions_state;
+}
+```
+
+### Namespace level
+
+1. Add the field `Boolean replicate_subscriptions_state` to the `org.apache.pulsar.common.policies.data.Policies` class
+   to control subscription replication at the namespace level:
+    ```java
+    public class Policies {
+        @SuppressWarnings("checkstyle:MemberName")
+        public Boolean replicate_subscription_state;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.Namespaces` interface:
+     ```java
+     public interface Namespaces {
+         void setReplicateSubscriptionState(String namespace, Boolean enabled) throws PulsarAdminException;
+         CompletableFuture<Void> setReplicateSubscriptionStateAsync(String namespace, Boolean enabled);
+         Boolean getReplicateSubscriptionState(String namespace) throws PulsarAdminException;
+         CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String namespace);
+     }
+     ```
+3. Implement the management methods in the `org.apache.pulsar.client.admin.internal.NamespacesImpl` class.
+
+### Topic level
+
+1. Add the field `Boolean replicateSubscriptionState` to the `org.apache.pulsar.common.policies.data.TopicPolicies`
+   class to enable subscription replication at the topic level:
+   ```java
+    public class TopicPolicies {
+         public Boolean replicateSubscriptionState;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.TopicPolicies` interface:
+   ```java
+       public interface TopicPolicies { 
+           void setReplicateSubscriptionState(String topic, Boolean enabled) throws PulsarAdminException;
+           CompletableFuture<Void> setReplicateSubscriptionStateAsync(String topic, Boolean enabled);
+           Boolean getReplicateSubscriptionState(String topic, boolean applied) throws PulsarAdminException;
+           CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String topic, boolean applied);
+       }

Review Comment:
   > We have a `pulsar-admin topics set-replicated-subscription-status` command, the current implementation uses Boolean for similar that, so I followed the same design for consistency and to avoid introducing a new pattern in the CLI/API.
   
   This is a valid argument if we are able to achieve all required use cases with a `Boolean`. 
   
   After thinking about it again, it seems that there are 3 different choices (instead of 4) that would need to be covered:
   
   1. override any setting by the consumer (or lower level policy) and enable subscription replication for all subscriptions
   2. override any setting by the consumer (or lower level policy) and disable subscription replication for all subscriptions
   3. enable subscription replication if not explicitly disabled by the consumer (or lower level policy) for all subscriptions
   
   @nodece With a single `Boolean` value, how would you distinguish between the 2 different cases (1. and 3. above) where subscription replication is enabled?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the broker, namespace and topic levels [pulsar]

Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on code in PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#discussion_r2304792666


##########
pip/pip-398.md:
##########
@@ -0,0 +1,253 @@
+# PIP-398: Subscription replication on the broker, namespace and topic levels
+
+# Background knowledge
+
+https://github.com/apache/pulsar/pull/4299 introduces the subscription replication feature on the consumer level:
+
+```java
+Consumer<byte[]> consumer = pulsarClient.newConsumer().topic("topic").replicateSubscriptionState(false/*true*/)
+        .subscriptionName("sub").subscribe();
+```
+
+While this provides flexibility, it introduces overhead in managing replication for a large number of consumers. Users
+need to manually enable the `replicateSubscriptionState` flag for each consumer, which can become cumbersome in
+large-scale deployments.
+
+# Motivation
+
+The key motivation behind this PIP is to simplify subscription replication configuration, especially in failover
+scenarios. When a main cluster goes down and a backup cluster is activated, ensuring that subscription states are
+consistently replicated across clusters is critical for failover scenarios. By extending the replication configuration
+to the broker, namespace and topic levels, the system reduces the need for explicit consumer-level configuration.
+
+# Goals
+
+## In Scope
+
+The PIP aims to provide management of subscription replication at the broker, namespace and topic levels using the
+Pulsar Admin CLI and API.
+
+# High Level Design
+
+Introduces the `replicateSubscriptionState` configuration to enabling subscription replication on the broker, namespace
+and topic levels, when enabled, all consumers under the broker/namespace/topic will automatically replicate their
+subscription states to remote clusters.
+
+The priority for the subscription replication configuration is as follows:
+
+- consumer level > topic level > namespace level > broker level.
+- If `replicateSubscriptionState` is set at the consumer level, configurations at the topic, namespace, and broker levels are
+  ignored.
+- If set at the topic level, the namespace-level configuration is ignored.
+- If set at the namespace level, the broker-level configuration is ignored.
+
+# Detailed Design
+
+## Design & Implementation Details
+
+### Broker level
+
+Add the field `Boolean replicate_subscriptions_state` to the `org.apache.pulsar.broker.ServiceConfiguration` class
+to control subscription replication at the broker level:
+```java
+public class ServiceConfiguration implements PulsarConfiguration {
+    @FieldContext(
+            category = CATEGORY_SERVER,
+            required = false,
+            dynamic = true,
+            doc = "The default value for replicating subscription state."
+    )
+    private Boolean replicate_subscriptions_state;
+}
+```
+
+### Namespace level
+
+1. Add the field `Boolean replicate_subscriptions_state` to the `org.apache.pulsar.common.policies.data.Policies` class
+   to control subscription replication at the namespace level:
+    ```java
+    public class Policies {
+        @SuppressWarnings("checkstyle:MemberName")
+        public Boolean replicate_subscription_state;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.Namespaces` interface:
+     ```java
+     public interface Namespaces {
+         void setReplicateSubscriptionState(String namespace, Boolean enabled) throws PulsarAdminException;
+         CompletableFuture<Void> setReplicateSubscriptionStateAsync(String namespace, Boolean enabled);
+         Boolean getReplicateSubscriptionState(String namespace) throws PulsarAdminException;
+         CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String namespace);
+     }
+     ```
+3. Implement the management methods in the `org.apache.pulsar.client.admin.internal.NamespacesImpl` class.
+
+### Topic level
+
+1. Add the field `Boolean replicateSubscriptionState` to the `org.apache.pulsar.common.policies.data.TopicPolicies`
+   class to enable subscription replication at the topic level:
+   ```java
+    public class TopicPolicies {
+         public Boolean replicateSubscriptionState;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.TopicPolicies` interface:
+   ```java
+       public interface TopicPolicies { 
+           void setReplicateSubscriptionState(String topic, Boolean enabled) throws PulsarAdminException;
+           CompletableFuture<Void> setReplicateSubscriptionStateAsync(String topic, Boolean enabled);
+           Boolean getReplicateSubscriptionState(String topic, boolean applied) throws PulsarAdminException;
+           CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String topic, boolean applied);
+       }

Review Comment:
   > We have a `pulsar-admin topics set-replicated-subscription-status` command, the current implementation uses Boolean for similar that, so I followed the same design for consistency and to avoid introducing a new pattern in the CLI/API.
   
   This is a valid argument if we are able to achieve all required use cases with a `Boolean`. 
   
   After thinking about it again, it seems that these choices need to be covered:
   
   1. override any setting by the consumer (or lower level policy) and enable subscription replication for all subscriptions
   2. override any setting by the consumer (or lower level policy) and disable subscription replication for all subscriptions
   3. enable subscription replication if not explicitly disabled by the consumer (or lower level policy) for all subscriptions
   4. no setting at all (higher level setting would take priority). If all policies (broker, ns, topic) are "empty", the consumer setting would get used directly.
   
   @nodece With a single `Boolean` value, how would you distinguish between the 2 different cases (1. and 3. above) where subscription replication is enabled?
   
   (with "lower level policy", it means the hierarchy of broker -> namespace -> topic)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the broker, namespace and topic levels [pulsar]

Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on code in PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#discussion_r2910428918


##########
pip/pip-398.md:
##########
@@ -0,0 +1,253 @@
+# PIP-398: Subscription replication on the broker, namespace and topic levels
+
+# Background knowledge
+
+https://github.com/apache/pulsar/pull/4299 introduces the subscription replication feature on the consumer level:
+
+```java
+Consumer<byte[]> consumer = pulsarClient.newConsumer().topic("topic").replicateSubscriptionState(false/*true*/)
+        .subscriptionName("sub").subscribe();
+```
+
+While this provides flexibility, it introduces overhead in managing replication for a large number of consumers. Users
+need to manually enable the `replicateSubscriptionState` flag for each consumer, which can become cumbersome in
+large-scale deployments.
+
+# Motivation
+
+The key motivation behind this PIP is to simplify subscription replication configuration, especially in failover
+scenarios. When a main cluster goes down and a backup cluster is activated, ensuring that subscription states are
+consistently replicated across clusters is critical for failover scenarios. By extending the replication configuration
+to the broker, namespace and topic levels, the system reduces the need for explicit consumer-level configuration.
+
+# Goals
+
+## In Scope
+
+The PIP aims to provide management of subscription replication at the broker, namespace and topic levels using the
+Pulsar Admin CLI and API.
+
+# High Level Design
+
+Introduces the `replicateSubscriptionState` configuration to enabling subscription replication on the broker, namespace
+and topic levels, when enabled, all consumers under the broker/namespace/topic will automatically replicate their
+subscription states to remote clusters.
+
+The priority for the subscription replication configuration is as follows:
+
+- consumer level > topic level > namespace level > broker level.
+- If `replicateSubscriptionState` is set at the consumer level, configurations at the topic, namespace, and broker levels are
+  ignored.
+- If set at the topic level, the namespace-level configuration is ignored.
+- If set at the namespace level, the broker-level configuration is ignored.

Review Comment:
   @nodece Please update this to the PIP document too. It's good to explain the "what", "how" and "why" as you already did in your answer above. Having this in the PIP document will be it clear to everyone.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the broker, namespace and topic levels [pulsar]

Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on code in PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#discussion_r2905153552


##########
pip/pip-398.md:
##########
@@ -0,0 +1,253 @@
+# PIP-398: Subscription replication on the broker, namespace and topic levels
+
+# Background knowledge
+
+https://github.com/apache/pulsar/pull/4299 introduces the subscription replication feature on the consumer level:
+
+```java
+Consumer<byte[]> consumer = pulsarClient.newConsumer().topic("topic").replicateSubscriptionState(false/*true*/)
+        .subscriptionName("sub").subscribe();
+```
+
+While this provides flexibility, it introduces overhead in managing replication for a large number of consumers. Users
+need to manually enable the `replicateSubscriptionState` flag for each consumer, which can become cumbersome in
+large-scale deployments.
+
+# Motivation
+
+The key motivation behind this PIP is to simplify subscription replication configuration, especially in failover
+scenarios. When a main cluster goes down and a backup cluster is activated, ensuring that subscription states are
+consistently replicated across clusters is critical for failover scenarios. By extending the replication configuration
+to the broker, namespace and topic levels, the system reduces the need for explicit consumer-level configuration.
+
+# Goals
+
+## In Scope
+
+The PIP aims to provide management of subscription replication at the broker, namespace and topic levels using the
+Pulsar Admin CLI and API.
+
+# High Level Design
+
+Introduces the `replicateSubscriptionState` configuration to enabling subscription replication on the broker, namespace
+and topic levels, when enabled, all consumers under the broker/namespace/topic will automatically replicate their
+subscription states to remote clusters.
+
+The priority for the subscription replication configuration is as follows:
+
+- consumer level > topic level > namespace level > broker level.
+- If `replicateSubscriptionState` is set at the consumer level, configurations at the topic, namespace, and broker levels are
+  ignored.
+- If set at the topic level, the namespace-level configuration is ignored.
+- If set at the namespace level, the broker-level configuration is ignored.

Review Comment:
   Based on the discussion thread at https://github.com/apache/pulsar/pull/23770/changes#r2905108272 , it seems that the PIP considers the case where the setting is "set and enabled (Boolean.TRUE)".
   I misunderstood the design since a value could be considered "set" when it's `Boolean.FALSE`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the broker, namespace and topic levels [pulsar]

Posted by "nodece (via GitHub)" <gi...@apache.org>.
nodece commented on code in PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#discussion_r2909764921


##########
pip/pip-398.md:
##########
@@ -0,0 +1,253 @@
+# PIP-398: Subscription replication on the broker, namespace and topic levels
+
+# Background knowledge
+
+https://github.com/apache/pulsar/pull/4299 introduces the subscription replication feature on the consumer level:
+
+```java
+Consumer<byte[]> consumer = pulsarClient.newConsumer().topic("topic").replicateSubscriptionState(false/*true*/)
+        .subscriptionName("sub").subscribe();
+```
+
+While this provides flexibility, it introduces overhead in managing replication for a large number of consumers. Users
+need to manually enable the `replicateSubscriptionState` flag for each consumer, which can become cumbersome in
+large-scale deployments.
+
+# Motivation
+
+The key motivation behind this PIP is to simplify subscription replication configuration, especially in failover
+scenarios. When a main cluster goes down and a backup cluster is activated, ensuring that subscription states are
+consistently replicated across clusters is critical for failover scenarios. By extending the replication configuration
+to the broker, namespace and topic levels, the system reduces the need for explicit consumer-level configuration.
+
+# Goals
+
+## In Scope
+
+The PIP aims to provide management of subscription replication at the broker, namespace and topic levels using the
+Pulsar Admin CLI and API.
+
+# High Level Design
+
+Introduces the `replicateSubscriptionState` configuration to enabling subscription replication on the broker, namespace
+and topic levels, when enabled, all consumers under the broker/namespace/topic will automatically replicate their
+subscription states to remote clusters.
+
+The priority for the subscription replication configuration is as follows:
+
+- consumer level > topic level > namespace level > broker level.
+- If `replicateSubscriptionState` is set at the consumer level, configurations at the topic, namespace, and broker levels are
+  ignored.
+- If set at the topic level, the namespace-level configuration is ignored.
+- If set at the namespace level, the broker-level configuration is ignored.

Review Comment:
   @lhotari You are right.
   
   The replicated flag at the subscription level is persisted in BookKeeper cursor metadata, and only `true` indicates a replicated subscription.
   
   To keep the semantics consistent across subscription, topic, and namespace
   levels, the policy only treats `Boolean.TRUE` as enabling the feature. `false` or `null` both mean "not enabling it".
   
   Otherwise, the topic could end up in inconsistent states where a subscription is replicated while the topic policy explicitly disables this.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the broker, namespace and topic levels [pulsar]

Posted by "nodece (via GitHub)" <gi...@apache.org>.
nodece commented on code in PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#discussion_r1959583186


##########
pip/pip-398.md:
##########
@@ -0,0 +1,245 @@
+# PIP-398: Subscription replication on the broker, namespace and topic levels
+
+# Background knowledge
+
+https://github.com/apache/pulsar/pull/4299 introduces the subscription replication feature on the consumer level:
+
+```java
+Consumer<byte[]> consumer = pulsarClient.newConsumer().topic("topic").replicateSubscriptionState(false/*true*/)
+        .subscriptionName("sub").subscribe();
+```
+
+While this provides flexibility, it introduces overhead in managing replication for a large number of consumers. Users
+need to manually enable the `replicateSubscriptionState` flag for each consumer, which can become cumbersome in
+large-scale deployments.
+
+# Motivation
+
+The key motivation behind this PIP is to simplify subscription replication configuration, especially in failover
+scenarios. When a main cluster goes down and a backup cluster is activated, ensuring that subscription states are
+consistently replicated across clusters is critical for failover scenarios. By extending the replication configuration
+to the broker, namespace and topic levels, the system reduces the need for explicit consumer-level configuration.
+
+# Goals
+
+## In Scope
+
+The PIP aims to provide management of subscription replication at the broker, namespace and topic levels using the
+Pulsar Admin CLI and API.
+
+# High Level Design
+
+Introduces a configuration used for enabling subscription replication on the broker, namespace and topic levels, when
+enabled, all consumers under the broker/namespace/topic will automatically replicate their subscription states to remote
+clusters.
+
+The priority for the subscription replication configuration is as follows:
+
+- consumer level > topic level > namespace level > broker level.
+- If `replicateSubscriptionState` is set at the consumer level, configurations at the topic, namespace, and broker levels are
+  ignored.
+- If set at the topic level, the namespace-level configuration is ignored.
+- If set at the namespace level, the broker-level configuration is ignored.

Review Comment:
   > I think that there could be a need to override any value set on the consumer side.
   
   It's also possible.  
   
   > I also added some comments that controlling the behavior at namespace and consumer level should perhaps be a policy concern instead of creating a separate concept for handling the setting.
   
   Yes. This config is present in the broker/namespace policies/topic policies/cursor property.
   
   > If the setting at namespace and topic level would be more than just a true/false. It would be possible to add a separate setting that would allow overriding the consumer level setting.
   
   I don't want to introduce more config, so when the consume level is false, the broker/namespace policies/topic policies can override the consume level setting.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the namespace and topic levels [pulsar]

Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#issuecomment-2559116084

   @nodece would it be possible to cover #23769 feature too? That enables subscription replication for all subscriptions at the broker level if I understand it correctly. Could you work together with @yyj8 to cover that?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the namespace and topic levels [pulsar]

Posted by "nodece (via GitHub)" <gi...@apache.org>.
nodece commented on PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#issuecomment-2565003158

   > Before using cluster level configuration, we first check if there are topic and namespace dimensions. If there are none, we apply cluster level configuration; The priority of the three dimensions from high to low is: topic -> namespace -> cluster.
   
   @yyj8 Correct. The `replicateSubscriptionState` is used to auto-create a subscription. When set to `false`, the broker doesn't persist that to the cursor property in the old version. Therefore, we only need to add `replicateSubscriptionState=true/false/null` at the cluster level.
   
   **Breaking Change**: For an existing subscription where `replicateSubscriptionState=false` is set at the consumer level, the new broker will use the subscription replication policy from the topic/namespace/cluster level.
   
   The new subscription will follow the above priority.
   
   /cc @lhotari


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [improve][pip] PIP-398: Subscription replication on the broker, namespace and topic levels [pulsar]

Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on code in PR #23770:
URL: https://github.com/apache/pulsar/pull/23770#discussion_r2297750795


##########
pip/pip-398.md:
##########
@@ -0,0 +1,253 @@
+# PIP-398: Subscription replication on the broker, namespace and topic levels
+
+# Background knowledge
+
+https://github.com/apache/pulsar/pull/4299 introduces the subscription replication feature on the consumer level:
+
+```java
+Consumer<byte[]> consumer = pulsarClient.newConsumer().topic("topic").replicateSubscriptionState(false/*true*/)
+        .subscriptionName("sub").subscribe();
+```
+
+While this provides flexibility, it introduces overhead in managing replication for a large number of consumers. Users
+need to manually enable the `replicateSubscriptionState` flag for each consumer, which can become cumbersome in
+large-scale deployments.
+
+# Motivation
+
+The key motivation behind this PIP is to simplify subscription replication configuration, especially in failover
+scenarios. When a main cluster goes down and a backup cluster is activated, ensuring that subscription states are
+consistently replicated across clusters is critical for failover scenarios. By extending the replication configuration
+to the broker, namespace and topic levels, the system reduces the need for explicit consumer-level configuration.
+
+# Goals
+
+## In Scope
+
+The PIP aims to provide management of subscription replication at the broker, namespace and topic levels using the
+Pulsar Admin CLI and API.
+
+# High Level Design
+
+Introduces the `replicateSubscriptionState` configuration to enabling subscription replication on the broker, namespace
+and topic levels, when enabled, all consumers under the broker/namespace/topic will automatically replicate their
+subscription states to remote clusters.
+
+The priority for the subscription replication configuration is as follows:
+
+- consumer level > topic level > namespace level > broker level.
+- If `replicateSubscriptionState` is set at the consumer level, configurations at the topic, namespace, and broker levels are
+  ignored.
+- If set at the topic level, the namespace-level configuration is ignored.
+- If set at the namespace level, the broker-level configuration is ignored.
+
+# Detailed Design
+
+## Design & Implementation Details
+
+### Broker level
+
+Add the field `Boolean replicate_subscriptions_state` to the `org.apache.pulsar.broker.ServiceConfiguration` class
+to control subscription replication at the broker level:
+```java
+public class ServiceConfiguration implements PulsarConfiguration {
+    @FieldContext(
+            category = CATEGORY_SERVER,
+            required = false,
+            dynamic = true,
+            doc = "The default value for replicating subscription state."
+    )
+    private Boolean replicate_subscriptions_state;
+}
+```
+
+### Namespace level
+
+1. Add the field `Boolean replicate_subscriptions_state` to the `org.apache.pulsar.common.policies.data.Policies` class
+   to control subscription replication at the namespace level:
+    ```java
+    public class Policies {
+        @SuppressWarnings("checkstyle:MemberName")
+        public Boolean replicate_subscription_state;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.Namespaces` interface:
+     ```java
+     public interface Namespaces {
+         void setReplicateSubscriptionState(String namespace, Boolean enabled) throws PulsarAdminException;
+         CompletableFuture<Void> setReplicateSubscriptionStateAsync(String namespace, Boolean enabled);
+         Boolean getReplicateSubscriptionState(String namespace) throws PulsarAdminException;
+         CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String namespace);
+     }
+     ```
+3. Implement the management methods in the `org.apache.pulsar.client.admin.internal.NamespacesImpl` class.
+
+### Topic level
+
+1. Add the field `Boolean replicateSubscriptionState` to the `org.apache.pulsar.common.policies.data.TopicPolicies`
+   class to enable subscription replication at the topic level:
+   ```java
+    public class TopicPolicies {
+         public Boolean replicateSubscriptionState;
+    }
+    ```
+2. Add the management methods to the `org.apache.pulsar.client.admin.TopicPolicies` interface:
+   ```java
+       public interface TopicPolicies { 
+           void setReplicateSubscriptionState(String topic, Boolean enabled) throws PulsarAdminException;
+           CompletableFuture<Void> setReplicateSubscriptionStateAsync(String topic, Boolean enabled);
+           Boolean getReplicateSubscriptionState(String topic, boolean applied) throws PulsarAdminException;
+           CompletableFuture<Boolean> getReplicateSubscriptionStateAsync(String topic, boolean applied);
+       }

Review Comment:
   I would strongly suggest replacing the `Boolean` with an enum. Using a boolean value for `replicateSubscriptionState` is very confusing.
   
   Here's an example what `ReplicateSubscriptionStatePolicy` enum could be:
   * `OVERRIDE_ENABLE` - overrides any value set by the consumer (or lower level policy) and enables subscription replication
   * `OVERRIDE_DISABLE` - overrides any value set by the consumer (or lower level policy) and disables subscription replication
   * `ENABLE` - enables subscription replication if not explicitly disabled by the consumer (or lower level policy)
   * `DISABLE` - disables subscription replication if not explicitly enabled by the consumer (or lower level policy)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org