You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pulsar.apache.org by 李鹏辉 <co...@gmail.com> on 2018/08/21 10:14:07 UTC
PIP 23 : Message Tracing By Interceptors
Dear all
This is a PIP to add feature of message trace implement by interceptors.
https://docs.google.com/document/d/1aigUKZ-AhQvugOfzUzypyMMeXfum5XlNkwuYK4KvxJs/edit?usp=sharing
—
Regards,
Penghui Li
Re: PIP 23 : Message Tracing By Interceptors
Posted by Sijie Guo <gu...@gmail.com>.
On Fri, Aug 24, 2018 at 6:04 PM Matteo Merli <ma...@gmail.com> wrote:
> > Original, i just want to implement tracing by interceptors. We don’t must
> > to binding “tracing” and “interceptors” together.
> > I thougt that to implement message tracing in pulsar, we should implement
> > interceptors before. And interceptors might be useful in multiple
> scenarios.
>
> Yes, that's exactly what I meant. Interceptors are a good feature to have
> in general and tracing can be implemented on top of that.
> I think we should probably separate the 2 aspects and have this proposal
> just focus on interceptors.
>
I think the PIP was meant to focus on interceptors (and optionally a dump
client tracing interceptor).
We can have a separate PIP for "Integrating with OpenTracing" and another
PIP for "Integrating with OpenCensus" in future.
So we are able to reduce the scope and move PIP forward much faster.
- Sijie
>
> One other thing to consider is that for tracing to be "meaningful" we would
> also need to have support in broker side (and, as you mentioned, ideally in
> bookkeeper as well). That means that either:
>
> * We provide "interceptors" hooks also for broker
> * or we have some "fixed" tracing implementation that we support.
>
> In any case, my previous example was to show how, as a user, I'd like the
> enabling tracing in Pulsar should be. Since tracing will be a very common
> feature, we should aim to have a tight integration with few common tracing
> providers, while leaving the freedom for user to integrate with custom
> tracing systems.
>
> Matteo
>
>
>
> On Fri, Aug 24, 2018 at 3:02 AM 李鹏辉 <co...@gmail.com> wrote:
>
> > Original, i just want to implement tracing by interceptors. We don’t must
> > to binding “tracing” and “interceptors” together.
> >
> > I thougt that to implement message tracing in pulsar, we should implement
> > interceptors before. And interceptors might be useful in multiple
> > scenarios.
> >
> --
> Matteo Merli
> <mm...@apache.org>
>
Re: PIP 23 : Message Tracing By Interceptors
Posted by Matteo Merli <ma...@gmail.com>.
> Original, i just want to implement tracing by interceptors. We don’t must
> to binding “tracing” and “interceptors” together.
> I thougt that to implement message tracing in pulsar, we should implement
> interceptors before. And interceptors might be useful in multiple
scenarios.
Yes, that's exactly what I meant. Interceptors are a good feature to have
in general and tracing can be implemented on top of that.
I think we should probably separate the 2 aspects and have this proposal
just focus on interceptors.
One other thing to consider is that for tracing to be "meaningful" we would
also need to have support in broker side (and, as you mentioned, ideally in
bookkeeper as well). That means that either:
* We provide "interceptors" hooks also for broker
* or we have some "fixed" tracing implementation that we support.
In any case, my previous example was to show how, as a user, I'd like the
enabling tracing in Pulsar should be. Since tracing will be a very common
feature, we should aim to have a tight integration with few common tracing
providers, while leaving the freedom for user to integrate with custom
tracing systems.
Matteo
On Fri, Aug 24, 2018 at 3:02 AM 李鹏辉 <co...@gmail.com> wrote:
> Original, i just want to implement tracing by interceptors. We don’t must
> to binding “tracing” and “interceptors” together.
>
> I thougt that to implement message tracing in pulsar, we should implement
> interceptors before. And interceptors might be useful in multiple
> scenarios.
>
--
Matteo Merli
<mm...@apache.org>
Re: PIP 23 : Message Tracing By Interceptors
Posted by 李鹏辉 <co...@gmail.com>.
Original, i just want to implement tracing by interceptors. We don’t must
to binding “tracing” and “interceptors” together.
I thougt that to implement message tracing in pulsar, we should implement
interceptors before. And interceptors might be useful in multiple scenarios.
Re: PIP 23 : Message Tracing By Interceptors
Posted by Sijie Guo <gu...@gmail.com>.
On Thu, Aug 23, 2018 at 11:25 PM Matteo Merli <ma...@gmail.com>
wrote:
> Hi Penghui,
>
> the proposal looks good to me. I like the approach of using interceptors
> since they might be useful in multiple scenarios, for applications to add
> custom logging or processing.
>
> For tracing, in particular, I think we should also aim to have "out of the
> box" ready made solution as well, especially targeting standard APIs like
> OpenTracing. We shouldn't be dictating any particular implementation, but
> we should try to avoid the burden of integrating with tracing APIs to
> application developers.
> Ideally, as a user, I should be able to do something like :
>
> client.newConsumer()
> .topic("xyz")
> .subscriptionName("yyz")
> .withTracing(OpenTracing.of(tracingLoggerConfig))
>
Why can't we use the proposed #intercept(...)? I am not sure we need to
distinguish "tracing" from other interceptors.
client.newConsumer()
.topic("xyz")
.subscriptionName("yyz")
.intercept(OpenTracingInterceptor.of(tracingLoggerConfig))
> .subscribe();
>
> (Or maybe even configure that at the PulsarClient level)
>
>
> On Tue, Aug 21, 2018 at 11:13 AM Sijie Guo <gu...@gmail.com> wrote:
>
> > Updated. Thanks, Penghui!
> >
> > - Sijie
> >
> > On Tue, Aug 21, 2018 at 3:56 AM 李鹏辉 <co...@gmail.com> wrote:
> >
> > > Please add zhaijia to Authors.
> > >
> > > Sijie Guo <gu...@gmail.com> 于2018年8月21日周二 下午6:48写道：
> > >
> > > > Thank you penghui! This is a great feature to add to Pulsar.
> > > >
> > > > I've copied your PIP to pulsar wiki as well :
> > > >
> > > >
> > >
> >
> https://github.com/apache/incubator-pulsar/wiki/PIP-23:-Message-Tracing-By-Interceptors
> > > >
> > > > - Sijie
> > > >
> > > > On Tue, Aug 21, 2018 at 3:14 AM 李鹏辉 <co...@gmail.com> wrote:
> > > >
> > > > > Dear all
> > > > >
> > > > > This is a PIP to add feature of message trace implement by
> > > interceptors.
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> https://docs.google.com/document/d/1aigUKZ-AhQvugOfzUzypyMMeXfum5XlNkwuYK4KvxJs/edit?usp=sharing
> > > > >
> > > > > —
> > > > > Regards,
> > > > > Penghui Li
> > > > >
> > > >
> > >
> >
> --
> Matteo Merli
> <mm...@apache.org>
>
Re: PIP 23 : Message Tracing By Interceptors
Posted by Matteo Merli <ma...@gmail.com>.
Hi Penghui,
the proposal looks good to me. I like the approach of using interceptors
since they might be useful in multiple scenarios, for applications to add
custom logging or processing.
For tracing, in particular, I think we should also aim to have "out of the
box" ready made solution as well, especially targeting standard APIs like
OpenTracing. We shouldn't be dictating any particular implementation, but
we should try to avoid the burden of integrating with tracing APIs to
application developers.
Ideally, as a user, I should be able to do something like :
client.newConsumer()
.topic("xyz")
.subscriptionName("yyz")
.withTracing(OpenTracing.of(tracingLoggerConfig))
.subscribe();
(Or maybe even configure that at the PulsarClient level)
On Tue, Aug 21, 2018 at 11:13 AM Sijie Guo <gu...@gmail.com> wrote:
> Updated. Thanks, Penghui!
>
> - Sijie
>
> On Tue, Aug 21, 2018 at 3:56 AM 李鹏辉 <co...@gmail.com> wrote:
>
> > Please add zhaijia to Authors.
> >
> > Sijie Guo <gu...@gmail.com> 于2018年8月21日周二 下午6:48写道：
> >
> > > Thank you penghui! This is a great feature to add to Pulsar.
> > >
> > > I've copied your PIP to pulsar wiki as well :
> > >
> > >
> >
> https://github.com/apache/incubator-pulsar/wiki/PIP-23:-Message-Tracing-By-Interceptors
> > >
> > > - Sijie
> > >
> > > On Tue, Aug 21, 2018 at 3:14 AM 李鹏辉 <co...@gmail.com> wrote:
> > >
> > > > Dear all
> > > >
> > > > This is a PIP to add feature of message trace implement by
> > interceptors.
> > > >
> > > >
> > > >
> > >
> >
> https://docs.google.com/document/d/1aigUKZ-AhQvugOfzUzypyMMeXfum5XlNkwuYK4KvxJs/edit?usp=sharing
> > > >
> > > > —
> > > > Regards,
> > > > Penghui Li
> > > >
> > >
> >
>
--
Matteo Merli
<mm...@apache.org>
Re: PIP 23 : Message Tracing By Interceptors
Posted by Sijie Guo <gu...@gmail.com>.
Updated. Thanks, Penghui!
- Sijie
On Tue, Aug 21, 2018 at 3:56 AM 李鹏辉 <co...@gmail.com> wrote:
> Please add zhaijia to Authors.
>
> Sijie Guo <gu...@gmail.com> 于2018年8月21日周二 下午6:48写道：
>
> > Thank you penghui! This is a great feature to add to Pulsar.
> >
> > I've copied your PIP to pulsar wiki as well :
> >
> >
> https://github.com/apache/incubator-pulsar/wiki/PIP-23:-Message-Tracing-By-Interceptors
> >
> > - Sijie
> >
> > On Tue, Aug 21, 2018 at 3:14 AM 李鹏辉 <co...@gmail.com> wrote:
> >
> > > Dear all
> > >
> > > This is a PIP to add feature of message trace implement by
> interceptors.
> > >
> > >
> > >
> >
> https://docs.google.com/document/d/1aigUKZ-AhQvugOfzUzypyMMeXfum5XlNkwuYK4KvxJs/edit?usp=sharing
> > >
> > > —
> > > Regards,
> > > Penghui Li
> > >
> >
>
Re: PIP 23 : Message Tracing By Interceptors
Posted by 李鹏辉 <co...@gmail.com>.
Please add zhaijia to Authors.
Sijie Guo <gu...@gmail.com> 于2018年8月21日周二 下午6:48写道：
> Thank you penghui! This is a great feature to add to Pulsar.
>
> I've copied your PIP to pulsar wiki as well :
>
> https://github.com/apache/incubator-pulsar/wiki/PIP-23:-Message-Tracing-By-Interceptors
>
> - Sijie
>
> On Tue, Aug 21, 2018 at 3:14 AM 李鹏辉 <co...@gmail.com> wrote:
>
> > Dear all
> >
> > This is a PIP to add feature of message trace implement by interceptors.
> >
> >
> >
> https://docs.google.com/document/d/1aigUKZ-AhQvugOfzUzypyMMeXfum5XlNkwuYK4KvxJs/edit?usp=sharing
> >
> > —
> > Regards,
> > Penghui Li
> >
>
Re: PIP 23 : Message Tracing By Interceptors
Posted by Sijie Guo <gu...@gmail.com>.
Thank you penghui! This is a great feature to add to Pulsar.
I've copied your PIP to pulsar wiki as well :
https://github.com/apache/incubator-pulsar/wiki/PIP-23:-Message-Tracing-By-Interceptors
- Sijie
On Tue, Aug 21, 2018 at 3:14 AM 李鹏辉 <co...@gmail.com> wrote:
> Dear all
>
> This is a PIP to add feature of message trace implement by interceptors.
>
>
> https://docs.google.com/document/d/1aigUKZ-AhQvugOfzUzypyMMeXfum5XlNkwuYK4KvxJs/edit?usp=sharing
>
> —
> Regards,
> Penghui Li
>