You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/05/07 14:30:50 UTC

[GitHub] [pulsar] eolivelli opened a new pull request, #15488: [enh] Exclusive Producer: ability to fence out an existing Producer (ExclusiveWithFencing mode)

eolivelli opened a new pull request, #15488:
URL: https://github.com/apache/pulsar/pull/15488

   ### Motivation
   Currently (in 2.10) we have two ways of creating a Exclusive Producer: Exclusive and WaitForExclusive.
   We are missing a mode in which the new Producer fences out any existing producer, allowing to apply Optimistic Locking pattern. 
   
   ### Modifications
   
   - Add a new mode ExclusiveWithFencing
   - Add tests
   - Implementation on the Broker side
   - Implementation on the Java Client
   
   ### Verifying this change
   
   This change added tests.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] merlimat commented on a diff in pull request #15488: [enh] Exclusive Producer: ability to fence out an existing Producer (ExclusiveWithFencing mode)

Posted by GitBox <gi...@apache.org>.
merlimat commented on code in PR #15488:
URL: https://github.com/apache/pulsar/pull/15488#discussion_r867379604


##########
pulsar-broker/src/test/java/org/apache/pulsar/broker/service/ExclusiveProducerTest.java:
##########
@@ -118,7 +120,22 @@ private void simpleTest(String topic) throws Exception {
                 .topic(topic)
                 .accessMode(ProducerAccessMode.Exclusive)
                 .create();
-        p2.close();
+
+        Producer<String> p3 = pulsarClient.newProducer(Schema.STRING)
+                .topic(topic)
+                .accessMode(ProducerAccessMode.ExclusiveWithFencing)
+                .create();
+
+        try {
+            p2.send("test");
+            fail("Should have failed");
+        } catch (ProducerFencedException expected) {
+        }
+
+        // this should work
+        p3.send("test");
+        p3.close();

Review Comment:
   I'd also add a test to verify that any queued up producer is also immediately fenced. eg: mixing up `WaitForExclusive` & `ExclusiveWithFencing`



##########
pulsar-client-api/src/main/java/org/apache/pulsar/client/api/ProducerAccessMode.java:
##########
@@ -33,6 +33,11 @@ public enum ProducerAccessMode {
      */
     Exclusive,
 
+    /**
+     * Require exclusive access for producer. Fence out the old producer.

Review Comment:
   I think many people might not be super familiar with the word "fencing" :)
   
   ```suggestion
        * Acquire exclusive access for the producer. Any existing producer will be immediately
        * fenced off and will not be able to publish anymore.
   ```



##########
pulsar-client-api/src/main/java/org/apache/pulsar/client/api/ProducerBuilder.java:
##########
@@ -133,6 +133,8 @@
      * <li>{@link ProducerAccessMode#Shared}: By default multiple producers can publish on a topic
      * <li>{@link ProducerAccessMode#Exclusive}: Require exclusive access for producer. Fail immediately if there's
      * already a producer connected.
+     * <li>{@link ProducerAccessMode#ExclusiveWithFencing}: Require exclusive access for producer. Fence out any
+     * producer that is connected.

Review Comment:
   ```suggestion
        * <li>{@link ProducerAccessMode#ExclusiveWithFencing}: Acquire exclusive access for the producer. 
        * Any existing producer will be immediately fenced off and will not be able to publish anymore.
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] github-actions[bot] commented on pull request #15488: [enh] Exclusive Producer: ability to fence out an existing Producer (ExclusiveWithFencing mode)

Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #15488:
URL: https://github.com/apache/pulsar/pull/15488#issuecomment-1120219909

   @eolivelli:Thanks for your contribution. For this PR, do we need to update docs?
   (The [PR template contains info about doc](https://github.com/apache/pulsar/blob/master/.github/PULL_REQUEST_TEMPLATE.md#documentation), which helps others know more about the changes. Can you provide doc-related info in this and future PR descriptions? Thanks)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] Anonymitaet commented on pull request #15488: [enh] Exclusive Producer: ability to fence out an existing Producer (ExclusiveWithFencing mode)

Posted by GitBox <gi...@apache.org>.
Anonymitaet commented on PR #15488:
URL: https://github.com/apache/pulsar/pull/15488#issuecomment-1120554058

   @momo-jun a soft reminder: here is a PR w/ doc-required label, could u pls follow up? Thanks


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] eolivelli commented on pull request #15488: PIP-161 Exclusive Producer: ability to fence out an existing Producer (ExclusiveWithFencing mode)

Posted by GitBox <gi...@apache.org>.
eolivelli commented on PR #15488:
URL: https://github.com/apache/pulsar/pull/15488#issuecomment-1131511994

   /pulsarbot rerun-failure-checks


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] mattisonchao commented on pull request #15488: [enh] Exclusive Producer: ability to fence out an existing Producer (ExclusiveWithFencing mode)

Posted by GitBox <gi...@apache.org>.
mattisonchao commented on PR #15488:
URL: https://github.com/apache/pulsar/pull/15488#issuecomment-1120521871

   It changed the protocol and public API, I'm not sure if we need a proposal for it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] eolivelli commented on pull request #15488: PIP-161 Exclusive Producer: ability to fence out an existing Producer (ExclusiveWithFencing mode)

Posted by GitBox <gi...@apache.org>.
eolivelli commented on PR #15488:
URL: https://github.com/apache/pulsar/pull/15488#issuecomment-1123288286

   thank you @Anonymitaet. I have committed your suggestion


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] eolivelli commented on pull request #15488: PIP-161 Exclusive Producer: ability to fence out an existing Producer (ExclusiveWithFencing mode)

Posted by GitBox <gi...@apache.org>.
eolivelli commented on PR #15488:
URL: https://github.com/apache/pulsar/pull/15488#issuecomment-1129797393

   /pulsarbot rerun-failure-checks


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] eolivelli commented on pull request #15488: PIP-161 Exclusive Producer: ability to fence out an existing Producer (ExclusiveWithFencing mode)

Posted by GitBox <gi...@apache.org>.
eolivelli commented on PR #15488:
URL: https://github.com/apache/pulsar/pull/15488#issuecomment-1129639248

   @merlimat please take another look


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] eolivelli commented on pull request #15488: PIP-161 Exclusive Producer: ability to fence out an existing Producer (ExclusiveWithFencing mode)

Posted by GitBox <gi...@apache.org>.
eolivelli commented on PR #15488:
URL: https://github.com/apache/pulsar/pull/15488#issuecomment-1131599866

   /pulsarbot rerun-failure-checks


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] eolivelli commented on pull request #15488: PIP-161 Exclusive Producer: ability to fence out an existing Producer (ExclusiveWithFencing mode)

Posted by GitBox <gi...@apache.org>.
eolivelli commented on PR #15488:
URL: https://github.com/apache/pulsar/pull/15488#issuecomment-1131684671

   /pulsarbot rerun-failure-checks


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] Anonymitaet commented on a diff in pull request #15488: PIP-161 Exclusive Producer: ability to fence out an existing Producer (ExclusiveWithFencing mode)

Posted by GitBox <gi...@apache.org>.
Anonymitaet commented on code in PR #15488:
URL: https://github.com/apache/pulsar/pull/15488#discussion_r869827850


##########
site2/docs/concepts-messaging.md:
##########
@@ -65,6 +65,7 @@ You can have different types of access modes on topics for producers.
 |---|---
 `Shared`|Multiple producers can publish on a topic. <br><br>This is the **default** setting.
 `Exclusive`|Only one producer can publish on a topic. <br><br>If there is already a producer connected, other producers trying to publish on this topic get errors immediately.<br><br>The “old” producer is evicted and a “new” producer is selected to be the next exclusive producer if the “old” producer experiences a network partition with the broker.
+`ExclusiveWithFencing`|Only one producer can publish on a topic. <br><br>If there is already a producer connected, is will be removed and invalidated immediately.

Review Comment:
   ```suggestion
   `ExclusiveWithFencing`|Only one producer can publish on a topic. <br /><br />If there is already a producer connected, it will be removed and invalidated immediately.
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] momo-jun commented on pull request #15488: [enh] Exclusive Producer: ability to fence out an existing Producer (ExclusiveWithFencing mode)

Posted by GitBox <gi...@apache.org>.
momo-jun commented on PR #15488:
URL: https://github.com/apache/pulsar/pull/15488#issuecomment-1121809206

   @eolivelli I can add docs for this PR as soon as it is approved/merged. If you'd like to do it, feel free to ping me for review. Either way works for me.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] eolivelli commented on pull request #15488: [enh] Exclusive Producer: ability to fence out an existing Producer (ExclusiveWithFencing mode)

Posted by GitBox <gi...@apache.org>.
eolivelli commented on PR #15488:
URL: https://github.com/apache/pulsar/pull/15488#issuecomment-1122342231

   > For the "doc-required" label, it's needed to remember updating the docs. eg: https://pulsar.apache.org/docs/en/concepts-messaging/#access-mode
   
   I have updated that section, thanks for the reminder.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] merlimat commented on a diff in pull request #15488: [enh] Exclusive Producer: ability to fence out an existing Producer (ExclusiveWithFencing mode)

Posted by GitBox <gi...@apache.org>.
merlimat commented on code in PR #15488:
URL: https://github.com/apache/pulsar/pull/15488#discussion_r868658827


##########
pulsar-client-api/src/main/java/org/apache/pulsar/client/api/ProducerAccessMode.java:
##########
@@ -33,6 +33,11 @@ public enum ProducerAccessMode {
      */
     Exclusive,
 
+    /**
+     * Acquire exclusive access for the producer. Any existing producer will be immediately.

Review Comment:
   ```suggestion
        * Acquire exclusive access for the producer. Any existing producer will be removed and
        * invalidated immediately.
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] eolivelli commented on a diff in pull request #15488: [enh] Exclusive Producer: ability to fence out an existing Producer (ExclusiveWithFencing mode)

Posted by GitBox <gi...@apache.org>.
eolivelli commented on code in PR #15488:
URL: https://github.com/apache/pulsar/pull/15488#discussion_r869188103


##########
pulsar-client-api/src/main/java/org/apache/pulsar/client/api/ProducerAccessMode.java:
##########
@@ -33,6 +33,11 @@ public enum ProducerAccessMode {
      */
     Exclusive,
 
+    /**
+     * Acquire exclusive access for the producer. Any existing producer will be immediately.

Review Comment:
   thanks



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] eolivelli merged pull request #15488: PIP-161 Exclusive Producer: ability to fence out an existing Producer (ExclusiveWithFencing mode)

Posted by GitBox <gi...@apache.org>.
eolivelli merged PR #15488:
URL: https://github.com/apache/pulsar/pull/15488


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org