You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@eventmesh.apache.org by GitBox <gi...@apache.org> on 2022/04/06 03:55:22 UTC

[GitHub] [incubator-eventmesh] AhahaGe opened a new issue, #825: [Feature] Event Filter

AhahaGe opened a new issue, #825:
URL: https://github.com/apache/incubator-eventmesh/issues/825

   ### Search before asking
   
   - [X] I had searched in the [issues](https://github.com/apache/eventmesh/issues?q=is%3Aissue) and found no similar issues.
   
   
   ### Feature Request
   
   1. Scenario
   Traditional, when consumer want to filter interested messages,  consumer need to rely on specified MQ client and code upon it to accomplish. But when it comes to apache EventMesh, users only need to **submit customized expressions which defines message filter logic to EventMesh**, EventMesh will do the filter and return filtered messages to users.
   
   2. Implementation
   
   2.1 Implementation One
   Do stream processing in EventMesh, it's MQ independent, but have some impact in performance. And we can maintain the expression engine ourself.
   - concrete implementation, define a set of EventMesh api to integrate and implement filter rule engine, combine with EQL and Json path. And we can also consider integrating the research rules engine (DataSonnet).
   
   2.2 Bind to RocketMq, use RocketMq as engine to realize filter.
   - concrete implementation, implement openmessaging-api interface, and use RocketMq to realize filter ability.
   
   - Conclusion: At present, it is inclined to plan 2.1. The ability of filter has nothing to do with specific MQ components. The specific plan still needs to be investigated.
   
   - related problems:
   1. Can Kafka originally support filter?
   2. Does RocketMq's filter support multiple thread, currently support.
   3. Consider multi-tenant segregation.
   4. The storage problem of rules supports plug-in implementation.
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@eventmesh.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@eventmesh.apache.org
For additional commands, e-mail: dev-help@eventmesh.apache.org


[GitHub] [incubator-eventmesh] jinrongluo commented on issue #825: [Feature] Event Filter

Posted by GitBox <gi...@apache.org>.
jinrongluo commented on issue #825:
URL: https://github.com/apache/incubator-eventmesh/issues/825#issuecomment-1099315094

   I have a quick look at Drools and EasyRule, their rules are not for filtering.  I think we need the simple rules like RocketMQ filters
   
   https://rocketmq.apache.org/docs/filter-by-sql92-example/
   
   for example:  $.subject == 'myservice'
   
   We can implement our own simple Rule engine to process this rules using Parse Tree (https://en.wikipedia.org/wiki/Parse_tree) and JsonPath SDK
   
   Concrete Implementation:
   
   1. Parse the Filter Expression into Parse Tree
   2. Evaluate each node of the Parse Tree using JsonPath, and Java API
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@eventmesh.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@eventmesh.apache.org
For additional commands, e-mail: dev-help@eventmesh.apache.org


[GitHub] [incubator-eventmesh] xwm1992 commented on issue #825: [Feature] Event Filter

Posted by GitBox <gi...@apache.org>.
xwm1992 commented on issue #825:
URL: https://github.com/apache/incubator-eventmesh/issues/825#issuecomment-1099922916

   > I have a quick look at Drools and EasyRule, their rules are not for filtering. I think we need the simple rules like RocketMQ filters
   > 
   > https://rocketmq.apache.org/docs/filter-by-sql92-example/
   > 
   > for example: $.subject == 'myservice'
   > 
   > We can implement our own simple Rule engine to process this rules using Parse Tree (https://en.wikipedia.org/wiki/Parse_tree) and JsonPath SDK
   > 
   > Concrete Implementation:
   > 
   > 1. Parse the Filter Expression into Parse Tree
   > 2. Evaluate each node of the Parse Tree using JsonPath, and Java API
   
   This is a good proposal,may be we can learn more about the `Parse Tree` and `JsonPath SDK`. If we want to implement our own rule engine another direction to consider is implement this under the `JSR 94`, looking forward to other people opinion.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@eventmesh.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@eventmesh.apache.org
For additional commands, e-mail: dev-help@eventmesh.apache.org


[GitHub] [incubator-eventmesh] AhahaGe commented on issue #825: [Feature] Event Filter

Posted by GitBox <gi...@apache.org>.
AhahaGe commented on issue #825:
URL: https://github.com/apache/incubator-eventmesh/issues/825#issuecomment-1090169799

   > Do we need to integrate the rules engine, such as Drools or EasyRule ?
   
   Sounds good. We can discuss later and I add it into problem list.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@eventmesh.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@eventmesh.apache.org
For additional commands, e-mail: dev-help@eventmesh.apache.org


[GitHub] [incubator-eventmesh] ruanwenjun commented on issue #825: [Feature] Event Filter

Posted by GitBox <gi...@apache.org>.
ruanwenjun commented on issue #825:
URL: https://github.com/apache/incubator-eventmesh/issues/825#issuecomment-1089798183

   Plan 2.1 look good to me. I think the performance will not be impacted a lot unless we need to do some join operation.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@eventmesh.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@eventmesh.apache.org
For additional commands, e-mail: dev-help@eventmesh.apache.org


Re: [I] [Feature] Event Filter (eventmesh)

Posted by "Pil0tXia (via GitHub)" <gi...@apache.org>.
Pil0tXia commented on issue #825:
URL: https://github.com/apache/eventmesh/issues/825#issuecomment-1884788036

   There are some good examples that can serve as references for the UI of the filter. For instance, Cloudflare's Transform Rules:
   
   ![image](https://github.com/apache/eventmesh/assets/41445332/1aab9252-83b5-46c2-8414-3f826716c1ad)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@eventmesh.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@eventmesh.apache.org
For additional commands, e-mail: issues-help@eventmesh.apache.org


[GitHub] [incubator-eventmesh] xwm1992 commented on issue #825: [Feature] Event Filter

Posted by GitBox <gi...@apache.org>.
xwm1992 commented on issue #825:
URL: https://github.com/apache/incubator-eventmesh/issues/825#issuecomment-1089932045

   Do we need to integrate the rules engine, such as Drools or EasyRule ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@eventmesh.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@eventmesh.apache.org
For additional commands, e-mail: dev-help@eventmesh.apache.org


[GitHub] [incubator-eventmesh] jinrongluo commented on issue #825: [Feature] Event Filter

Posted by GitBox <gi...@apache.org>.
jinrongluo commented on issue #825:
URL: https://github.com/apache/incubator-eventmesh/issues/825#issuecomment-1099793318

   
   I think SQL where syntax is  good enough. What do other people think?
   
   Alex 
   
   
   > On Apr 14, 2022, at 9:27 PM, AhahaGe ***@***.***> wrote:
   > 
   > 
   > Do u mean just like a sql where filter, where topic = 'myservice' and name='a' or age = '1', and what we need is a engine like a sql parse engine?
   > 
   > —
   > Reply to this email directly, view it on GitHub, or unsubscribe.
   > You are receiving this because you commented.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@eventmesh.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@eventmesh.apache.org
For additional commands, e-mail: dev-help@eventmesh.apache.org


Re: [I] [Feature] Event Filter (eventmesh)

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #825:
URL: https://github.com/apache/eventmesh/issues/825#issuecomment-2073120722

   It has been 90 days since the last activity on this issue. Apache EventMesh values the voices of the community. Please don't hesitate to share your latest insights on this matter at any time, as the community is more than willing to engage in discussions regarding the development and optimization directions of this feature.
   
   If you feel that your issue has been resolved, please feel free to close it. Should you have any additional information to share, you are welcome to reopen this issue.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@eventmesh.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@eventmesh.apache.org
For additional commands, e-mail: issues-help@eventmesh.apache.org


[GitHub] [incubator-eventmesh] AhahaGe commented on issue #825: [Feature] Event Filter

Posted by GitBox <gi...@apache.org>.
AhahaGe commented on issue #825:
URL: https://github.com/apache/incubator-eventmesh/issues/825#issuecomment-1099770543

   Do u mean just like a sql where filter, where topic = 'myservice' and name='a' or age = '1', and what we need is a engine like a sql parse engine?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@eventmesh.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@eventmesh.apache.org
For additional commands, e-mail: dev-help@eventmesh.apache.org