You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "tudormaftei (via GitHub)" <gi...@apache.org> on 2023/02/10 06:27:20 UTC

[GitHub] [pulsar] tudormaftei opened a new issue, #19482: Publish messages with properties from a WindowFunction

tudormaftei opened a new issue, #19482:
URL: https://github.com/apache/pulsar/issues/19482

   ### Search before asking
   
   - [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
   
   
   ### Motivation
   
   When using a **_WindowFunction_** to publish messages to an output topic, the original properties of the input messages are not forwarded. Forwarding properties works for functions which process individual messages but it doesn't work for functions which process a collection of messages.
   
   
   ### Solution
   
   A few possible solutions:
   
   1. Enhance **_WindowContext_** to be able to publish properties together with properties.
   
   Messages can be published to the output topic using the publish method of the **_WindowContext_** received as a parameter in **_WindowFunction.process_** method. Additional params could be added to the **_WindowContext.publish_** method to specify the properties to be included in the output message.
   
   2. Enhance the function configuration parameters.
   
   Additional configuration parameters can be added in the yaml function spec which could be used as properties for all messages published from the function.
   
   3. Add the possibility to publish messages of type **_Message<T>_** which already incorporate properties.
   
   
   ### Alternatives
   
   _No response_
   
   ### Anything else?
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [X] I'm willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] github-actions[bot] commented on issue #19482: Publish messages with properties from a WindowFunction

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #19482:
URL: https://github.com/apache/pulsar/issues/19482#issuecomment-1491180409

   The issue had no activity for 30 days, mark with Stale label.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Publish messages with properties from a WindowFunction [pulsar]

Posted by "merlimat (via GitHub)" <gi...@apache.org>.
merlimat closed issue #19482: Publish messages with properties from a WindowFunction
URL: https://github.com/apache/pulsar/issues/19482


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Publish messages with properties from a WindowFunction [pulsar]

Posted by "merlimat (via GitHub)" <gi...@apache.org>.
merlimat commented on issue #19482:
URL: https://github.com/apache/pulsar/issues/19482#issuecomment-6388835116

   Thanks @tudormaftei. `WindowContext` still has only `publish(topic, object)` and `publish(topic, object, schemaOrSerde)`, with no way to set properties (https://github.com/apache/pulsar/blob/master/pulsar-functions/api-java/src/main/java/org/apache/pulsar/functions/api/WindowContext.java). Each of the proposed options, a `newOutputMessage()` builder on `WindowContext` (as `Context` has) or new function config fields, changes the public Functions API, so it needs a PIP: https://github.com/apache/pulsar/blob/master/pip/README.md. Since you offered to contribute, a short PIP for adding `newOutputMessage` to `WindowContext` would be welcome. I'm closing this for now.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org