You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@rocketmq.apache.org by Rick Zhang <zh...@gmail.com> on 2017/03/16 22:31:00 UTC
Is extracting some source codes from one apache project to another
one appropriate?
Hi Bruce, RocketMQ Team,
RocketMQ Team hopes to add the functionality that support message filter
based on a subset of SQL92 expression syntax [1]. As everyone knows, JMS
Spec already defined a set of selector syntax , which is widely known by
people around java world. In particular, open source like ActiveMQ has
already implemented it very well (ActiveMQ also extends it to support
XPATH).
My question: Is it appropriate that extracting ActiveMQ's JMS selector
implementation into RocketMQ codebase(maybe we will add some extensions
upon it)?
I have no idea that if this behavior is correct. I just feel do the same
thing again may be a waste of time, especially the performance of JavaCC
used by ActiveMQ is good enough, and I have no sufficient reason to rewrite
a brand new one.
Looking forward you to give us some suggestions.
Thanks
[1] https://issues.apache.org/jira/browse/ROCKETMQ-121
Re: Is extracting some source codes from one apache project to
another one appropriate?
Posted by liuxue <li...@gmail.com>.
Hi,
As I know, RocketMQ only support filter message by tag which one message
could own one. It's too simple and not enough to meet some customized
scenarios.
So RockeMQ should have the ability to filter message through some
reasonable languages, such as SQL92, AVIATOR(from google) and any others,
no matter which one.
I think the ActiveMQ's solution which define the SQL92 grammar with javaCC
is simple and scalable, so could integrate all or part grammars into
RocketMQ.
Just as Von Gosling said, we could make a comment in the code, which is
learn from ActiveMQ.
Later, we also could integrate aviator or any other module into RocketMQ to
support more features.
Thanks,
Eric
On Fri, Mar 17, 2017 at 9:51 AM, Von Gosling <vo...@apache.org> wrote:
> Hi,
>
>
> There is no doubt that the feature is a great polish for RocketMQ filter
> implementation, which is using a proxy mechanism. So, IMO, we can use the
> experience of ActiveMQ’s solution - SQL 92 grammar with JavaCC compiling.
> But, we must make a comment in the code, which is learn from ActiveMQ -
> just like other Apache project similar practice. In fact, some guys have
> contact us privately, they have done this work. We encourage them to make
> discussion in dev list. Believe, if they have seen this mail, he must be
> greatly happy to make work together. Let’s polish together, making RocketMQ
> filter become more practical :-）
>
>
> > 在 2017年3月17日，06:31，Rick Zhang <zh...@gmail.com> 写道：
> >
> > Hi Bruce, RocketMQ Team,
> >
> > RocketMQ Team hopes to add the functionality that support message filter
> > based on a subset of SQL92 expression syntax [1]. As everyone knows, JMS
> > Spec already defined a set of selector syntax , which is widely known by
> > people around java world. In particular, open source like ActiveMQ has
> > already implemented it very well (ActiveMQ also extends it to support
> > XPATH).
> >
> > My question: Is it appropriate that extracting ActiveMQ's JMS selector
> > implementation into RocketMQ codebase(maybe we will add some extensions
> > upon it)?
> >
> > I have no idea that if this behavior is correct. I just feel do the same
> > thing again may be a waste of time, especially the performance of JavaCC
> > used by ActiveMQ is good enough, and I have no sufficient reason to
> rewrite
> > a brand new one.
> >
> > Looking forward you to give us some suggestions.
> >
> > Thanks
> >
> > [1] https://issues.apache.org/jira/browse/ROCKETMQ-121
>
>
Re: Is extracting some source codes from one apache project to
another one appropriate?
Posted by Von Gosling <vo...@apache.org>.
Hi,
There is no doubt that the feature is a great polish for RocketMQ filter implementation, which is using a proxy mechanism. So, IMO, we can use the experience of ActiveMQ’s solution - SQL 92 grammar with JavaCC compiling. But, we must make a comment in the code, which is learn from ActiveMQ - just like other Apache project similar practice. In fact, some guys have contact us privately, they have done this work. We encourage them to make discussion in dev list. Believe, if they have seen this mail, he must be greatly happy to make work together. Let’s polish together, making RocketMQ filter become more practical :-）
> 在 2017年3月17日，06:31，Rick Zhang <zh...@gmail.com> 写道：
>
> Hi Bruce, RocketMQ Team,
>
> RocketMQ Team hopes to add the functionality that support message filter
> based on a subset of SQL92 expression syntax [1]. As everyone knows, JMS
> Spec already defined a set of selector syntax , which is widely known by
> people around java world. In particular, open source like ActiveMQ has
> already implemented it very well (ActiveMQ also extends it to support
> XPATH).
>
> My question: Is it appropriate that extracting ActiveMQ's JMS selector
> implementation into RocketMQ codebase(maybe we will add some extensions
> upon it)?
>
> I have no idea that if this behavior is correct. I just feel do the same
> thing again may be a waste of time, especially the performance of JavaCC
> used by ActiveMQ is good enough, and I have no sufficient reason to rewrite
> a brand new one.
>
> Looking forward you to give us some suggestions.
>
> Thanks
>
> [1] https://issues.apache.org/jira/browse/ROCKETMQ-121
Re: Is extracting some source codes from one apache project to
another one appropriate?
Posted by Von Gosling <vo...@apache.org>.
Hi
Sorry to forget to CC user list :-) Repost ~
There is no doubt that the feature is a great polish for RocketMQ filter implementation, which is using a proxy mechanism. So, IMO, we can use the experience of ActiveMQ’s solution - SQL 92 grammar with JavaCC compiling. But, we must make a comment in the code, which is learn from ActiveMQ - just like other Apache project similar practice. In fact, some guys have contact us privately, they have done this work. We encourage them to make discussion in dev list. Believe, if they have seen this mail, he must be greatly happy to make work together. Let’s polish together, making RocketMQ filter become more practical :-）
> 在 2017年3月17日，06:31，Rick Zhang <zh...@gmail.com> 写道：
>
> Hi Bruce, RocketMQ Team,
>
> RocketMQ Team hopes to add the functionality that support message filter
> based on a subset of SQL92 expression syntax [1]. As everyone knows, JMS
> Spec already defined a set of selector syntax , which is widely known by
> people around java world. In particular, open source like ActiveMQ has
> already implemented it very well (ActiveMQ also extends it to support
> XPATH).
>
> My question: Is it appropriate that extracting ActiveMQ's JMS selector
> implementation into RocketMQ codebase(maybe we will add some extensions
> upon it)?
>
> I have no idea that if this behavior is correct. I just feel do the same
> thing again may be a waste of time, especially the performance of JavaCC
> used by ActiveMQ is good enough, and I have no sufficient reason to rewrite
> a brand new one.
>
> Looking forward you to give us some suggestions.
>
> Thanks
>
> [1] https://issues.apache.org/jira/browse/ROCKETMQ-121
Re: Is extracting some source codes from one apache project to
another one appropriate?
Posted by Rick Zhang <zh...@gmail.com>.
The collaboration suggestion is really helpful to me. Thanks all of you
very much!
If extracting ActiveMQ's SQL92 implement of message selector to an
individual module is feasible, IMO, it's the best way.
How do RocketMQ Team think about the suggestion?
2017-03-17 13:02 GMT+08:00 Bruce Snyder <br...@gmail.com>:
> Taking code instead of some sort of collaboration is not in the spirit of
> The Apache Way: http://theapacheway.com/
>
> Here's any idea -- how about working with the ActiveMQ project to extract
> its SQL92/XPATH implementation of message selectors into some sort of
> modular component that could be used on both projects? I am willing to
> introduce the idea if on the ActiveMQ dev list if you are interested.
>
> Bruce
>
> On Thu, Mar 16, 2017 at 8:45 PM, Justin Mclean <ju...@classsoftware.com>
> wrote:
>
> > Hi,
> >
> > > My question: Is it appropriate that extracting ActiveMQ's JMS selector
> > > implementation into RocketMQ codebase(maybe we will add some extensions
> > > upon it)?
> >
> > It not considered good form to copy other projects code and put into our
> > repo.
> >
> > It would be better to collaborate on the code in their project if that is
> > possible.
> >
> > Thanks,
> > Justin
> >
> >
>
>
> --
> perl -e 'print
> unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*" );'
>
> ActiveMQ in Action: http://bit.ly/2je6cQ
> Blog: http://bsnyder.org/ <http://bruceblog.org/>
> Twitter: http://twitter.com/brucesnyder
>
Re: Is extracting some source codes from one apache project to
another one appropriate?
Posted by Von Gosling <vo...@apache.org>.
I put the anther thread topic about the RocketMQ filter PR into this one, Let us to resolve it together.
Thanks Eric, we will review it as far as possible :-)
> Hi,
>
> I have already create a PR( https://github.com/apache/incubator-rocketmq/pull/82) , to support message filtering based on SQL92, just as previous email noted.
>
> Maybe the solution is not the best one, and also the code may have mistake.
>
> I expect members could check the solution and review the code.
>
>
> Thanks,
> Eric
> I have already implemented a draft edition to support filter message.The
> module of SQL92 grammar is extracted from ActiveMQ with only some grammars,
> such as
>
> 1. numeric comparison, like >, >=, <,<=, BETWEEN
> 2. character comparison, like =, <>, IN
> 3. check NULL
> 4. logical AND, logical OR, logical NOT
>
> And optimize some code, e.g. number threshold check, types check, and etc.
>
> Others, such as Function, XPath is not integrated.
>
> I have also consider about other language may be added later, the module is
> an independent jar and is nothing to do with RocketMQ. Other language
> labeled 'some_type' can register to this module and could be used by
> RocketMQ.
>
> To do message filter when pull message from store is the basic
> implementation.But for high subscription ratio (one topic and N consumers
> with SQL92 expression), it would increase the gc frequency and occupy more
> system resources, especially when message body is not small(greater than
> 1k). The reason is:
>
> 1. off-heap to heap, once every consumer subscribed same topic pull message.
> 2. decode message properties, once every consumer subscribed same topic
> pull message.
>
>
> So I optimize via calculate the filter results of all consumers subscribed
> same topic when broker build message queue(maybe slow down this procedure),
> and save the results to extend file linked to queue. I would describe the
> details in PR later.
>
> Thanks,
> Eric
>
>
> On Fri, Mar 17, 2017 at 2:52 PM, Von Gosling <vo...@apache.org> wrote:
>
>>> Here's any idea -- how about working with the ActiveMQ project to extract
>>> its SQL92/XPATH implementation of message selectors into some sort of
>>> modular component that could be used on both projects? I am willing to
>>> introduce the idea if on the ActiveMQ dev list if you are interested.
>>
>> Cool ~ If the filter module can be extracted from ActiveMQ, we are happy
>> to integrate it naturally. Let ’s listen the filter module contributor
>> liuxue’s idea. We want to learn more about some keys about his
>> implementation :-)
>>
>>
>>
>>> 在 2017年3月17日，13:02，Bruce Snyder <br...@gmail.com> 写道：
>>>
>>> Taking code instead of some sort of collaboration is not in the spirit of
>>> The Apache Way: http://theapacheway.com/
>>>
>>> Here's any idea -- how about working with the ActiveMQ project to extract
>>> its SQL92/XPATH implementation of message selectors into some sort of
>>> modular component that could be used on both projects? I am willing to
>>> introduce the idea if on the ActiveMQ dev list if you are interested.
>>>
>>> Bruce
>>>
>>> On Thu, Mar 16, 2017 at 8:45 PM, Justin Mclean <justin@classsoftware.com
>>>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>>> My question: Is it appropriate that extracting ActiveMQ's JMS selector
>>>>> implementation into RocketMQ codebase(maybe we will add some extensions
>>>>> upon it)?
>>>>
>>>> It not considered good form to copy other projects code and put into our
>>>> repo.
>>>>
>>>> It would be better to collaborate on the code in their project if that
>> is
>>>> possible.
>>>>
>>>> Thanks,
>>>> Justin
>>>>
>>>>
>>>
>>>
>>> --
>>> perl -e 'print
>>> unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*" );'
>>>
>>> ActiveMQ in Action: http://bit.ly/2je6cQ
>>> Blog: http://bsnyder.org/ <http://bruceblog.org/>
>>> Twitter: http://twitter.com/brucesnyder
>>
>>
Re: Is extracting some source codes from one apache project to
another one appropriate?
Posted by liuxue <li...@gmail.com>.
Hi,
I have already implemented a draft edition to support filter message.The
module of SQL92 grammar is extracted from ActiveMQ with only some grammars,
such as
1. numeric comparison, like >, >=, <,<=, BETWEEN
2. character comparison, like =, <>, IN
3. check NULL
4. logical AND, logical OR, logical NOT
And optimize some code, e.g. number threshold check, types check, and etc.
Others, such as Function, XPath is not integrated.
I have also consider about other language may be added later, the module is
an independent jar and is nothing to do with RocketMQ. Other language
labeled 'some_type' can register to this module and could be used by
RocketMQ.
To do message filter when pull message from store is the basic
implementation.But for high subscription ratio (one topic and N consumers
with SQL92 expression), it would increase the gc frequency and occupy more
system resources, especially when message body is not small(greater than
1k). The reason is:
1. off-heap to heap, once every consumer subscribed same topic pull message.
2. decode message properties, once every consumer subscribed same topic
pull message.
So I optimize via calculate the filter results of all consumers subscribed
same topic when broker build message queue(maybe slow down this procedure),
and save the results to extend file linked to queue. I would describe the
details in PR later.
Thanks,
Eric
On Fri, Mar 17, 2017 at 2:52 PM, Von Gosling <vo...@apache.org> wrote:
> > Here's any idea -- how about working with the ActiveMQ project to extract
> > its SQL92/XPATH implementation of message selectors into some sort of
> > modular component that could be used on both projects? I am willing to
> > introduce the idea if on the ActiveMQ dev list if you are interested.
>
> Cool ~ If the filter module can be extracted from ActiveMQ, we are happy
> to integrate it naturally. Let ’s listen the filter module contributor
> liuxue’s idea. We want to learn more about some keys about his
> implementation :-)
>
>
>
> > 在 2017年3月17日，13:02，Bruce Snyder <br...@gmail.com> 写道：
> >
> > Taking code instead of some sort of collaboration is not in the spirit of
> > The Apache Way: http://theapacheway.com/
> >
> > Here's any idea -- how about working with the ActiveMQ project to extract
> > its SQL92/XPATH implementation of message selectors into some sort of
> > modular component that could be used on both projects? I am willing to
> > introduce the idea if on the ActiveMQ dev list if you are interested.
> >
> > Bruce
> >
> > On Thu, Mar 16, 2017 at 8:45 PM, Justin Mclean <justin@classsoftware.com
> >
> > wrote:
> >
> >> Hi,
> >>
> >>> My question: Is it appropriate that extracting ActiveMQ's JMS selector
> >>> implementation into RocketMQ codebase(maybe we will add some extensions
> >>> upon it)?
> >>
> >> It not considered good form to copy other projects code and put into our
> >> repo.
> >>
> >> It would be better to collaborate on the code in their project if that
> is
> >> possible.
> >>
> >> Thanks,
> >> Justin
> >>
> >>
> >
> >
> > --
> > perl -e 'print
> > unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*" );'
> >
> > ActiveMQ in Action: http://bit.ly/2je6cQ
> > Blog: http://bsnyder.org/ <http://bruceblog.org/>
> > Twitter: http://twitter.com/brucesnyder
>
>
Re: Is extracting some source codes from one apache project to
another one appropriate?
Posted by Von Gosling <vo...@apache.org>.
> Here's any idea -- how about working with the ActiveMQ project to extract
> its SQL92/XPATH implementation of message selectors into some sort of
> modular component that could be used on both projects? I am willing to
> introduce the idea if on the ActiveMQ dev list if you are interested.
Cool ~ If the filter module can be extracted from ActiveMQ, we are happy to integrate it naturally. Let ’s listen the filter module contributor liuxue’s idea. We want to learn more about some keys about his implementation :-)
> 在 2017年3月17日，13:02，Bruce Snyder <br...@gmail.com> 写道：
>
> Taking code instead of some sort of collaboration is not in the spirit of
> The Apache Way: http://theapacheway.com/
>
> Here's any idea -- how about working with the ActiveMQ project to extract
> its SQL92/XPATH implementation of message selectors into some sort of
> modular component that could be used on both projects? I am willing to
> introduce the idea if on the ActiveMQ dev list if you are interested.
>
> Bruce
>
> On Thu, Mar 16, 2017 at 8:45 PM, Justin Mclean <ju...@classsoftware.com>
> wrote:
>
>> Hi,
>>
>>> My question: Is it appropriate that extracting ActiveMQ's JMS selector
>>> implementation into RocketMQ codebase(maybe we will add some extensions
>>> upon it)?
>>
>> It not considered good form to copy other projects code and put into our
>> repo.
>>
>> It would be better to collaborate on the code in their project if that is
>> possible.
>>
>> Thanks,
>> Justin
>>
>>
>
>
> --
> perl -e 'print
> unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*" );'
>
> ActiveMQ in Action: http://bit.ly/2je6cQ
> Blog: http://bsnyder.org/ <http://bruceblog.org/>
> Twitter: http://twitter.com/brucesnyder
Re: Is extracting some source codes from one apache project to
another one appropriate?
Posted by Bruce Snyder <br...@gmail.com>.
Taking code instead of some sort of collaboration is not in the spirit of
The Apache Way: http://theapacheway.com/
Here's any idea -- how about working with the ActiveMQ project to extract
its SQL92/XPATH implementation of message selectors into some sort of
modular component that could be used on both projects? I am willing to
introduce the idea if on the ActiveMQ dev list if you are interested.
Bruce
On Thu, Mar 16, 2017 at 8:45 PM, Justin Mclean <ju...@classsoftware.com>
wrote:
> Hi,
>
> > My question: Is it appropriate that extracting ActiveMQ's JMS selector
> > implementation into RocketMQ codebase(maybe we will add some extensions
> > upon it)?
>
> It not considered good form to copy other projects code and put into our
> repo.
>
> It would be better to collaborate on the code in their project if that is
> possible.
>
> Thanks,
> Justin
>
>
--
perl -e 'print
unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*" );'
ActiveMQ in Action: http://bit.ly/2je6cQ
Blog: http://bsnyder.org/ <http://bruceblog.org/>
Twitter: http://twitter.com/brucesnyder
Re: Is extracting some source codes from one apache project to
another one appropriate?
Posted by Justin Mclean <ju...@classsoftware.com>.
Hi,
> My question: Is it appropriate that extracting ActiveMQ's JMS selector
> implementation into RocketMQ codebase(maybe we will add some extensions
> upon it)?
It not considered good form to copy other projects code and put into our repo.
It would be better to collaborate on the code in their project if that is possible.
Thanks,
Justin
Re: Is extracting some source codes from one apache project to
another one appropriate?
Posted by Von Gosling <vo...@apache.org>.
Hi
Sorry to forget to CC user list :-) Repost ~
There is no doubt that the feature is a great polish for RocketMQ filter implementation, which is using a proxy mechanism. So, IMO, we can use the experience of ActiveMQ’s solution - SQL 92 grammar with JavaCC compiling. But, we must make a comment in the code, which is learn from ActiveMQ - just like other Apache project similar practice. In fact, some guys have contact us privately, they have done this work. We encourage them to make discussion in dev list. Believe, if they have seen this mail, he must be greatly happy to make work together. Let’s polish together, making RocketMQ filter become more practical :-）
> 在 2017年3月17日，06:31，Rick Zhang <zh...@gmail.com> 写道：
>
> Hi Bruce, RocketMQ Team,
>
> RocketMQ Team hopes to add the functionality that support message filter
> based on a subset of SQL92 expression syntax [1]. As everyone knows, JMS
> Spec already defined a set of selector syntax , which is widely known by
> people around java world. In particular, open source like ActiveMQ has
> already implemented it very well (ActiveMQ also extends it to support
> XPATH).
>
> My question: Is it appropriate that extracting ActiveMQ's JMS selector
> implementation into RocketMQ codebase(maybe we will add some extensions
> upon it)?
>
> I have no idea that if this behavior is correct. I just feel do the same
> thing again may be a waste of time, especially the performance of JavaCC
> used by ActiveMQ is good enough, and I have no sufficient reason to rewrite
> a brand new one.
>
> Looking forward you to give us some suggestions.
>
> Thanks
>
> [1] https://issues.apache.org/jira/browse/ROCKETMQ-121