You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/09/21 13:34:57 UTC

[GitHub] [pulsar] dionjansen opened a new issue, #17776: [Bug]

dionjansen opened a new issue, #17776:
URL: https://github.com/apache/pulsar/issues/17776

   ### Search before asking
   
   - [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
   
   
   ### Version
   
   Pulsar version: 2.10.1
   
   ### Minimal reproduce step
   
   Follow [Pulsar functions quickstart](https://github.com/apache/pulsar/blob/master/site2/website/versioned_docs/version-2.3.0/functions-quickstart.md#writing-and-running-a-new-function). I'm using Docker to run Pulsar standalone:
   
   ### 1. Run pulsar standalone
   
   First, create a new Python file:
   
   ```bash
   $ touch fn.py
   ```
   
   In that file, add the following:
   
   ```python
   def process(input: bytes):
       if "B" in str(input):
           raise Exception("Fail on 'B'")
       return input
   ```
   
   This function will raise an exception when the message context text contains the character `"B"`
   
   ### 2. Start the function with `--retain-ordering` flag
   
   ```bash
   $ docker exec -it pulsar bash -c "/pulsar/bin/pulsar-admin functions localrun \
     --py '/pulsar/fn.py' \
     --classname fn \
     --inputs persistent://public/default/input \
     --output persistent://public/default/output \
     --tenant public \
     --namespace default \
     --subs-name 'test' \
     --retain-ordering \
     --name fn"
   ```
   
   ### 3. Consume output topic
   
   ```bash
   $ docker exec -it pulsar bash -c "bin/pulsar-client consume \
     persistent://public/default/output \
     --subscription-name test \
     --num-messages 0"
   ```
   
   ### 3. Produce messages to the source topic containing `"B"`
   
   ```bash
   $ docker exec -it pulsar bash -c "/pulsar/bin/pulsar-client produce \
     persistent://public/default/input \
     --messages 'A,B,C,D,E,F'"
   ```
   
   ### What did you expect to see?
   
   The consumer in step (3) should output messages in the order that they were provided (so `A` and nothing more)
   
   ### What did you see instead?
   
   It continues processing messages after `B` (`C`, `D` ..) and thus does not retain the order anymore:
   
   ```bash
   ----- got message -----
   key:[null], properties:[__pfn_input_msg_id__=CAoQACAA, __pfn_input_topic__=persistent://public/default/input], content:A
   ----- got message -----
   key:[null], properties:[__pfn_input_msg_id__=CAoQAiAA, __pfn_input_topic__=persistent://public/default/input], content:C
   ----- got message -----
   key:[null], properties:[__pfn_input_msg_id__=CAoQAyAA, __pfn_input_topic__=persistent://public/default/input], content:D
   ----- got message -----
   key:[null], properties:[__pfn_input_msg_id__=CAoQBCAA, __pfn_input_topic__=persistent://public/default/input], content:E
   ----- got message -----
   key:[null], properties:[__pfn_input_msg_id__=CAoQBSAA, __pfn_input_topic__=persistent://public/default/input], content:F
   ```
   
   ### Anything else?
   
   Perhaps I do not understand the flag's description from the [functions-cli docs](https://pulsar.apache.org/docs/functions-cli):
   
   > retain-ordering: *Function consumes and processes messages in order.*`
   
   So I would expect that unhandled exceptions are taken into account to ensure that the messages are processed in the order they are sequenced in the source topic.
   
   ### Are you willing to submit a PR?
   
   - [ ] I'm willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Bug] python function does not respect `--retain-ordering` flag [pulsar]

Posted by "jiangpengcheng (via GitHub)" <gi...@apache.org>.

jiangpengcheng commented on issue #17776:
URL: https://github.com/apache/pulsar/issues/17776#issuecomment-1778447582

   msg which has an error will be nacked and then retried, functions even with the `--retain-ordering` will not wait for this failed msg to be succeeded but will continue to process other messages
   
   so the `--retain-ordering` just ensures the order of msgs that can be processed successfully
   
   you can try with a java function too:
   
   
   ```
   package org.example;
   
   import java.util.function.Function;
   
   public class JavaNativeExclamationFunction implements Function<String, String> {
       @Override
       public String apply(String input) {
           if (input.contains("B")) {
               throw new RuntimeException("fail on B");
           }
           return String.format("%s!", input);
       }
   }
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Bug] python function does not respect `--retain-ordering` flag [pulsar]

Posted by "tisonkun (via GitHub)" <gi...@apache.org>.

tisonkun commented on issue #17776:
URL: https://github.com/apache/pulsar/issues/17776#issuecomment-1785408727

   Closed as answred.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] github-actions[bot] commented on issue #17776: [Bug] python function does not respect `--retain-ordering` flag

Posted by GitBox <gi...@apache.org>.

github-actions[bot] commented on issue #17776:
URL: https://github.com/apache/pulsar/issues/17776#issuecomment-1291401785

   The issue had no activity for 30 days, mark with Stale label.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] github-actions[bot] commented on issue #17776: [Bug] python function does not respect `--retain-ordering` flag

Posted by GitBox <gi...@apache.org>.

github-actions[bot] commented on issue #17776:
URL: https://github.com/apache/pulsar/issues/17776#issuecomment-1328433351

   The issue had no activity for 30 days, mark with Stale label.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] dionjansen commented on issue #17776: [Bug] python function does not respect `--retain-ordering` flag

Posted by GitBox <gi...@apache.org>.

dionjansen commented on issue #17776:
URL: https://github.com/apache/pulsar/issues/17776#issuecomment-1294519739

   @mattisonchao @freezenet just wondering, is there any update on this? I would be willing to submit a PR, but I'd need some direction on what the approach for this should be. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] dionjansen commented on issue #17776: [Bug] python function does not respect `--retain-ordering` flag

Posted by "dionjansen (via GitHub)" <gi...@apache.org>.

dionjansen commented on issue #17776:
URL: https://github.com/apache/pulsar/issues/17776#issuecomment-1480013544

   @mattisonchao @freeznet @codelipenghui any update on this? 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] github-actions[bot] commented on issue #17776: [Bug] python function does not respect `--retain-ordering` flag

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.

github-actions[bot] commented on issue #17776:
URL: https://github.com/apache/pulsar/issues/17776#issuecomment-1518918854

   The issue had no activity for 30 days, mark with Stale label.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] mattisonchao commented on issue #17776: [Bug] python function does not respect `--retain-ordering` flag

Posted by GitBox <gi...@apache.org>.

mattisonchao commented on issue #17776:
URL: https://github.com/apache/pulsar/issues/17776#issuecomment-1257360687

   Hi @freeznet 
   Would you like to take a look at it?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Bug] python function does not respect `--retain-ordering` flag [pulsar]

Posted by "dionjansen (via GitHub)" <gi...@apache.org>.

dionjansen commented on issue #17776:
URL: https://github.com/apache/pulsar/issues/17776#issuecomment-1785385001

   > so the `--retain-ordering` just ensures the order of msgs that can be processed successfully
   
   @jiangpengcheng clear, then indeed I have misunderstood the description for this flag in the [functions-cli docs](https://pulsar.apache.org/docs/functions-cli), thank you for clarifying.
   
   I think this can be closed.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Bug] python function does not respect `--retain-ordering` flag [pulsar]

Posted by "tisonkun (via GitHub)" <gi...@apache.org>.

tisonkun closed issue #17776: [Bug] python function does not respect `--retain-ordering` flag
URL: https://github.com/apache/pulsar/issues/17776


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org