You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "heesung-sn (via GitHub)" <gi...@apache.org> on 2024/08/13 17:30:51 UTC

[PR] [pip] PIP-371 Support to specify message listeners in TableView constructor [pulsar]

heesung-sn opened a new pull request, #23167:
URL: https://github.com/apache/pulsar/pull/23167

   Please refer to this fork PR for an implementation example.  https://github.com/heesung-sn/pulsar/pull/79


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [pip] PIP-373 Support to specify message listeners in TableView constructor [pulsar]

Posted by "heesung-sn (via GitHub)" <gi...@apache.org>.
heesung-sn commented on code in PR #23167:
URL: https://github.com/apache/pulsar/pull/23167#discussion_r1717154424


##########
pip/pip-369.md:
##########
@@ -0,0 +1,89 @@
+# PIP-371: Support to specify message listeners in TableView constructor
+
+# Background knowledge
+
+### TableView 
+Ref: https://pulsar.apache.org/docs/next/concepts-clients/#tableview
+
+### TableView Listeners
+TableView runs the registered listeners on each received message. TableView users could utilize TableView Listeners for event-sourcing applications.
+
+### How to Register TableView Listeners
+
+Currently, `TableView#foreachAndListen` and `TableView#listen` interfaces are used to register listeners.   
+
+# Motivation
+issue : https://github.com/apache/pulsar/issues/23158
+
+As reported in the above issue, currently, the TableView listener registration methods, `foreachAndListen` and `listen`, are being called after `TableView#start()`. This timing issue can result in some messages being missed or not handled if the listeners are registered too late.
+
+Additionally, the current implementation of these listeners does not distinguish between existing messages (those present before the TableView started) and tail (future) messages (those received after the TableView started). Some business logic may require different handling for these two types of messages, but the current design does not support this differentiation.
+
+
+
+# Goals
+
+## In Scope
+
+- Support specifying TableView Listeners in TableView builder and constructor, so that TableView can register listeners as part of instance init and accordingly call them for all messages.
+- Support specifying TableView Listeners for existing and tail messages separately.
+- This design is based on Pulsar-Java-Client.
+
+# Detailed Design
+
+## Design & Implementation Details
+
+Add followings in TableViewBuilder.java
+```java
+    /**
+     * Set the message listeners.
+     * If {@link TableViewBuilder#existingMessageListeners} are not specified, 
+     * these listeners are used for both
+     * existing and tailing(future) messages in the topic.

Review Comment:
   Alternatively, we can introduce an enum TableViewMessageType{
   Existing,
   Tail
   }
   
   And expose this Api only 
   
   TableViewBuilder listeners(TableViewMessageType type, BiConsumer<String, T>... listeners)
   
   
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [pip] PIP-371 Support to specify message listeners in TableView constructor [pulsar]

Posted by "Denovo1998 (via GitHub)" <gi...@apache.org>.
Denovo1998 commented on PR #23167:
URL: https://github.com/apache/pulsar/pull/23167#issuecomment-2287411720

   PIP-371 and PIP-372 already exist.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [pip] PIP-371 Support to specify message listeners in TableView constructor [pulsar]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #23167:
URL: https://github.com/apache/pulsar/pull/23167#issuecomment-2286769536

   @heesung-sn Please add the following content to your PR description and select a checkbox:
   ```
   - [ ] `doc` <!-- Your PR contains doc changes -->
   - [ ] `doc-required` <!-- Your PR changes impact docs and you will update later -->
   - [ ] `doc-not-needed` <!-- Your PR changes do not impact docs -->
   - [ ] `doc-complete` <!-- Docs have been already added -->
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [pip] PIP-373 Support to specify message listeners in TableView constructor [pulsar]

Posted by "merlimat (via GitHub)" <gi...@apache.org>.
merlimat commented on PR #23167:
URL: https://github.com/apache/pulsar/pull/23167#issuecomment-6179360989

   Thanks for this proposal. A PIP is accepted through a vote on dev@pulsar.apache.org, and this one doesn't have a discussion or vote thread there. It has been open since August 2024. As part of the open-PR cleanup discussed on dev@, I'm closing PIP proposals that are more than six months old without a passed vote. If you'd like to move it forward, please reopen this PR and start a [DISCUSS] or [VOTE] thread on dev@. Note that PIP-373 has since been assigned to another proposal (pip/pip-373.md on master), so it would need a new number.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [pip] PIP-373 Support to specify message listeners in TableView constructor [pulsar]

Posted by "heesung-sn (via GitHub)" <gi...@apache.org>.
heesung-sn commented on code in PR #23167:
URL: https://github.com/apache/pulsar/pull/23167#discussion_r1717154424


##########
pip/pip-369.md:
##########
@@ -0,0 +1,89 @@
+# PIP-371: Support to specify message listeners in TableView constructor
+
+# Background knowledge
+
+### TableView 
+Ref: https://pulsar.apache.org/docs/next/concepts-clients/#tableview
+
+### TableView Listeners
+TableView runs the registered listeners on each received message. TableView users could utilize TableView Listeners for event-sourcing applications.
+
+### How to Register TableView Listeners
+
+Currently, `TableView#foreachAndListen` and `TableView#listen` interfaces are used to register listeners.   
+
+# Motivation
+issue : https://github.com/apache/pulsar/issues/23158
+
+As reported in the above issue, currently, the TableView listener registration methods, `foreachAndListen` and `listen`, are being called after `TableView#start()`. This timing issue can result in some messages being missed or not handled if the listeners are registered too late.
+
+Additionally, the current implementation of these listeners does not distinguish between existing messages (those present before the TableView started) and tail (future) messages (those received after the TableView started). Some business logic may require different handling for these two types of messages, but the current design does not support this differentiation.
+
+
+
+# Goals
+
+## In Scope
+
+- Support specifying TableView Listeners in TableView builder and constructor, so that TableView can register listeners as part of instance init and accordingly call them for all messages.
+- Support specifying TableView Listeners for existing and tail messages separately.
+- This design is based on Pulsar-Java-Client.
+
+# Detailed Design
+
+## Design & Implementation Details
+
+Add followings in TableViewBuilder.java
+```java
+    /**
+     * Set the message listeners.
+     * If {@link TableViewBuilder#existingMessageListeners} are not specified, 
+     * these listeners are used for both
+     * existing and tailing(future) messages in the topic.

Review Comment:
   Alternatively, we can introduce an enum TableViewMessageType{
   Existing,
   Tail
   }
   
   And expose this Api only 
   
   TableViewBuilder<T> messageListeners(TableViewMessageType type, BiConsumer<String, T>... listeners)
   
   
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [pip] PIP-373 Support to specify message listeners in TableView constructor [pulsar]

Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on PR #23167:
URL: https://github.com/apache/pulsar/pull/23167#issuecomment-2402301385

   @heesung-sn  please up the mailing list threads to the document.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [pip] PIP-373 Support to specify message listeners in TableView constructor [pulsar]

Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on code in PR #23167:
URL: https://github.com/apache/pulsar/pull/23167#discussion_r1716969399


##########
pip/pip-369.md:
##########
@@ -0,0 +1,89 @@
+# PIP-371: Support to specify message listeners in TableView constructor
+
+# Background knowledge
+
+### TableView 
+Ref: https://pulsar.apache.org/docs/next/concepts-clients/#tableview
+
+### TableView Listeners
+TableView runs the registered listeners on each received message. TableView users could utilize TableView Listeners for event-sourcing applications.
+
+### How to Register TableView Listeners
+
+Currently, `TableView#foreachAndListen` and `TableView#listen` interfaces are used to register listeners.   
+
+# Motivation
+issue : https://github.com/apache/pulsar/issues/23158
+
+As reported in the above issue, currently, the TableView listener registration methods, `foreachAndListen` and `listen`, are being called after `TableView#start()`. This timing issue can result in some messages being missed or not handled if the listeners are registered too late.
+
+Additionally, the current implementation of these listeners does not distinguish between existing messages (those present before the TableView started) and tail (future) messages (those received after the TableView started). Some business logic may require different handling for these two types of messages, but the current design does not support this differentiation.
+
+
+
+# Goals
+
+## In Scope
+
+- Support specifying TableView Listeners in TableView builder and constructor, so that TableView can register listeners as part of instance init and accordingly call them for all messages.
+- Support specifying TableView Listeners for existing and tail messages separately.
+- This design is based on Pulsar-Java-Client.
+
+# Detailed Design
+
+## Design & Implementation Details
+
+Add followings in TableViewBuilder.java
+```java
+    /**
+     * Set the message listeners.
+     * If {@link TableViewBuilder#existingMessageListeners} are not specified, 
+     * these listeners are used for both
+     * existing and tailing(future) messages in the topic.

Review Comment:
   I don't think that it's a great idea to use this for both types. If someone wants the listener on both types, why not just register it for the other type of listener separately?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [pip] PIP-373 Support to specify message listeners in TableView constructor [pulsar]

Posted by "heesung-sn (via GitHub)" <gi...@apache.org>.
heesung-sn commented on code in PR #23167:
URL: https://github.com/apache/pulsar/pull/23167#discussion_r1717144888


##########
pip/pip-369.md:
##########
@@ -0,0 +1,89 @@
+# PIP-371: Support to specify message listeners in TableView constructor
+
+# Background knowledge
+
+### TableView 
+Ref: https://pulsar.apache.org/docs/next/concepts-clients/#tableview
+
+### TableView Listeners
+TableView runs the registered listeners on each received message. TableView users could utilize TableView Listeners for event-sourcing applications.
+
+### How to Register TableView Listeners
+
+Currently, `TableView#foreachAndListen` and `TableView#listen` interfaces are used to register listeners.   
+
+# Motivation
+issue : https://github.com/apache/pulsar/issues/23158
+
+As reported in the above issue, currently, the TableView listener registration methods, `foreachAndListen` and `listen`, are being called after `TableView#start()`. This timing issue can result in some messages being missed or not handled if the listeners are registered too late.
+
+Additionally, the current implementation of these listeners does not distinguish between existing messages (those present before the TableView started) and tail (future) messages (those received after the TableView started). Some business logic may require different handling for these two types of messages, but the current design does not support this differentiation.
+
+
+
+# Goals
+
+## In Scope
+
+- Support specifying TableView Listeners in TableView builder and constructor, so that TableView can register listeners as part of instance init and accordingly call them for all messages.
+- Support specifying TableView Listeners for existing and tail messages separately.
+- This design is based on Pulsar-Java-Client.
+
+# Detailed Design
+
+## Design & Implementation Details
+
+Add followings in TableViewBuilder.java
+```java
+    /**
+     * Set the message listeners.
+     * If {@link TableViewBuilder#existingMessageListeners} are not specified, 
+     * these listeners are used for both
+     * existing and tailing(future) messages in the topic.

Review Comment:
   I dont have strong opinion here, but I thought that we want the client design for less code.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [pip] PIP-373 Support to specify message listeners in TableView constructor [pulsar]

Posted by "heesung-sn (via GitHub)" <gi...@apache.org>.
heesung-sn commented on code in PR #23167:
URL: https://github.com/apache/pulsar/pull/23167#discussion_r1717136497


##########
pip/pip-369.md:
##########
@@ -0,0 +1,89 @@
+# PIP-371: Support to specify message listeners in TableView constructor
+
+# Background knowledge
+
+### TableView 
+Ref: https://pulsar.apache.org/docs/next/concepts-clients/#tableview
+
+### TableView Listeners
+TableView runs the registered listeners on each received message. TableView users could utilize TableView Listeners for event-sourcing applications.
+
+### How to Register TableView Listeners
+
+Currently, `TableView#foreachAndListen` and `TableView#listen` interfaces are used to register listeners.   
+
+# Motivation
+issue : https://github.com/apache/pulsar/issues/23158
+
+As reported in the above issue, currently, the TableView listener registration methods, `foreachAndListen` and `listen`, are being called after `TableView#start()`. This timing issue can result in some messages being missed or not handled if the listeners are registered too late.
+
+Additionally, the current implementation of these listeners does not distinguish between existing messages (those present before the TableView started) and tail (future) messages (those received after the TableView started). Some business logic may require different handling for these two types of messages, but the current design does not support this differentiation.
+
+
+
+# Goals
+
+## In Scope
+
+- Support specifying TableView Listeners in TableView builder and constructor, so that TableView can register listeners as part of instance init and accordingly call them for all messages.
+- Support specifying TableView Listeners for existing and tail messages separately.
+- This design is based on Pulsar-Java-Client.
+
+# Detailed Design
+
+## Design & Implementation Details
+
+Add followings in TableViewBuilder.java
+```java
+    /**
+     * Set the message listeners.
+     * If {@link TableViewBuilder#existingMessageListeners} are not specified, 
+     * these listeners are used for both
+     * existing and tailing(future) messages in the topic.

Review Comment:
   That will require setting the same listener for both, which I think is redundant.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [pip] PIP-373 Support to specify message listeners in TableView constructor [pulsar]

Posted by "heesung-sn (via GitHub)" <gi...@apache.org>.
heesung-sn commented on code in PR #23167:
URL: https://github.com/apache/pulsar/pull/23167#discussion_r1717154424


##########
pip/pip-369.md:
##########
@@ -0,0 +1,89 @@
+# PIP-371: Support to specify message listeners in TableView constructor
+
+# Background knowledge
+
+### TableView 
+Ref: https://pulsar.apache.org/docs/next/concepts-clients/#tableview
+
+### TableView Listeners
+TableView runs the registered listeners on each received message. TableView users could utilize TableView Listeners for event-sourcing applications.
+
+### How to Register TableView Listeners
+
+Currently, `TableView#foreachAndListen` and `TableView#listen` interfaces are used to register listeners.   
+
+# Motivation
+issue : https://github.com/apache/pulsar/issues/23158
+
+As reported in the above issue, currently, the TableView listener registration methods, `foreachAndListen` and `listen`, are being called after `TableView#start()`. This timing issue can result in some messages being missed or not handled if the listeners are registered too late.
+
+Additionally, the current implementation of these listeners does not distinguish between existing messages (those present before the TableView started) and tail (future) messages (those received after the TableView started). Some business logic may require different handling for these two types of messages, but the current design does not support this differentiation.
+
+
+
+# Goals
+
+## In Scope
+
+- Support specifying TableView Listeners in TableView builder and constructor, so that TableView can register listeners as part of instance init and accordingly call them for all messages.
+- Support specifying TableView Listeners for existing and tail messages separately.
+- This design is based on Pulsar-Java-Client.
+
+# Detailed Design
+
+## Design & Implementation Details
+
+Add followings in TableViewBuilder.java
+```java
+    /**
+     * Set the message listeners.
+     * If {@link TableViewBuilder#existingMessageListeners} are not specified, 
+     * these listeners are used for both
+     * existing and tailing(future) messages in the topic.

Review Comment:
   Alternatively, we can introduce TableViewMessageType enum.
   ```java
   enum TableViewMessageType {
       Existing,
       Tail
   }
   ```
   
   Then, we can expose this Api only too.
   ```java
   TableViewBuilder<T> messageListeners(TableViewMessageType type, BiConsumer<String, T>... listeners)
   ```
   
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [pip] PIP-373 Support to specify message listeners in TableView constructor [pulsar]

Posted by "heesung-sn (via GitHub)" <gi...@apache.org>.
heesung-sn commented on code in PR #23167:
URL: https://github.com/apache/pulsar/pull/23167#discussion_r1717154424


##########
pip/pip-369.md:
##########
@@ -0,0 +1,89 @@
+# PIP-371: Support to specify message listeners in TableView constructor
+
+# Background knowledge
+
+### TableView 
+Ref: https://pulsar.apache.org/docs/next/concepts-clients/#tableview
+
+### TableView Listeners
+TableView runs the registered listeners on each received message. TableView users could utilize TableView Listeners for event-sourcing applications.
+
+### How to Register TableView Listeners
+
+Currently, `TableView#foreachAndListen` and `TableView#listen` interfaces are used to register listeners.   
+
+# Motivation
+issue : https://github.com/apache/pulsar/issues/23158
+
+As reported in the above issue, currently, the TableView listener registration methods, `foreachAndListen` and `listen`, are being called after `TableView#start()`. This timing issue can result in some messages being missed or not handled if the listeners are registered too late.
+
+Additionally, the current implementation of these listeners does not distinguish between existing messages (those present before the TableView started) and tail (future) messages (those received after the TableView started). Some business logic may require different handling for these two types of messages, but the current design does not support this differentiation.
+
+
+
+# Goals
+
+## In Scope
+
+- Support specifying TableView Listeners in TableView builder and constructor, so that TableView can register listeners as part of instance init and accordingly call them for all messages.
+- Support specifying TableView Listeners for existing and tail messages separately.
+- This design is based on Pulsar-Java-Client.
+
+# Detailed Design
+
+## Design & Implementation Details
+
+Add followings in TableViewBuilder.java
+```java
+    /**
+     * Set the message listeners.
+     * If {@link TableViewBuilder#existingMessageListeners} are not specified, 
+     * these listeners are used for both
+     * existing and tailing(future) messages in the topic.

Review Comment:
   Alternatively, we can introduce TableViewMessageType enum.
   ```java
   enum TableViewMessageType {
       Existing,
       Tail
   }
   ```
   
   Then, we can expose this Api only too.
   ```java
   TableViewBuilder<T> messageListeners(TableViewMessageType type, BiConsumer<String, T>... listeners)
   ```
   
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [pip] PIP-373 Support to specify message listeners in TableView constructor [pulsar]

Posted by "merlimat (via GitHub)" <gi...@apache.org>.
merlimat closed pull request #23167: [pip] PIP-373 Support to specify message listeners in TableView constructor
URL: https://github.com/apache/pulsar/pull/23167


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org