You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Michal Borowiecki <mi...@openbet.com> on 2017/06/03 08:23:08 UTC
Sink Processor definition
Hi all,
Streams docs say:
> * *Sink Processor*: A sink processor is a special type of stream
> processor that does not have down-stream processors. It sends any
> received records from its up-stream processors to a specified
> Kafka topic.
>
Would a processor that doesn't produce to a kafka topic (directly) but
only updates a state store also be considered a sink processor? I think yes.
I'll submit a PR to that effect unless I hear otherwise.
Cheers,
Michał
--
Signature
<http://www.openbet.com/> Michal Borowiecki
Senior Software Engineer L4
T: +44 208 742 1600
+44 203 249 8448
E: michal.borowiecki@openbet.com
W: www.openbet.com <http://www.openbet.com/>
OpenBet Ltd
Chiswick Park Building 9
566 Chiswick High Rd
London
W4 5XT
UK
<https://www.openbet.com/email_promo>
This message is confidential and intended only for the addressee. If you
have received this message in error, please immediately notify the
postmaster@openbet.com <ma...@openbet.com> and delete it
from your system as well as any copies. The content of e-mails as well
as traffic data may be monitored by OpenBet for employment and security
purposes. To protect the environment please do not print this e-mail
unless necessary. OpenBet Ltd. Registered Office: Chiswick Park Building
9, 566 Chiswick High Road, London, W4 5XT, United Kingdom. A company
registered in England and Wales. Registered no. 3134634. VAT no.
GB927523612
Re: Sink Processor definition
Posted by "Matthias J. Sax" <ma...@confluent.io>.
I think "sink" it the correct term here. It means that we write to a topic.
Processors, that don't have downstream nodes are called "terminal
operation" (at least in the DSL). Thus, a sink is also a "terminal
operation" but not the other way round.
So the docs are not optimal, as they put the "terminal" part into the
focus while the "writing to a topic" part is the main thing here.
-Matthias
On 6/3/17 1:52 AM, Michal Borowiecki wrote:
> Yes, I think the key distinction, from the point of view of that
> documentation section, is that it doesn't have downstream processors.
>
>
> On 03/06/17 09:48, Damian Guy wrote:
>> Hi Michal,
>>
>> In this case Sink Processor is really referring to a SinkNode that can
>> only produce to a kafka topic. Maybe the terminology is incorrect as
>> strictly speaking a processor that writes data to anything could be
>> considered a Sink Processor.
>>
>> On Sat, 3 Jun 2017 at 09:23 Michal Borowiecki
>> <michal.borowiecki@openbet.com <ma...@openbet.com>>
>> wrote:
>>
>> Hi all,
>>
>> Streams docs say:
>>
>>> * *Sink Processor*: A sink processor is a special type of
>>> stream processor that does not have down-stream processors.
>>> It sends any received records from its up-stream processors
>>> to a specified Kafka topic.
>>>
>> Would a processor that doesn't produce to a kafka topic (directly)
>> but only updates a state store also be considered a sink
>> processor? I think yes.
>>
>> I'll submit a PR to that effect unless I hear otherwise.
>>
>> Cheers,
>>
>> Michał
>>
>> --
>> <http://www.openbet.com/> Michal Borowiecki
>> Senior Software Engineer L4
>> T: +44 208 742 1600 <tel:+44%2020%208742%201600>
>>
>>
>> +44 203 249 8448 <tel:+44%2020%203249%208448>
>>
>>
>>
>> E: michal.borowiecki@openbet.com
>> <ma...@openbet.com>
>> W: www.openbet.com <http://www.openbet.com/>
>>
>>
>> OpenBet Ltd
>>
>> Chiswick Park Building 9
>>
>> 566 Chiswick High Rd
>>
>> London
>>
>> W4 5XT
>>
>> UK
>>
>>
>> <https://www.openbet.com/email_promo>
>>
>> This message is confidential and intended only for the addressee.
>> If you have received this message in error, please immediately
>> notify the postmaster@openbet.com <ma...@openbet.com>
>> and delete it from your system as well as any copies. The content
>> of e-mails as well as traffic data may be monitored by OpenBet for
>> employment and security purposes. To protect the environment
>> please do not print this e-mail unless necessary. OpenBet Ltd.
>> Registered Office: Chiswick Park Building 9, 566 Chiswick High
>> Road, London, W4 5XT, United Kingdom. A company registered in
>> England and Wales. Registered no. 3134634. VAT no. GB927523612
>>
>
> --
> Signature
> <http://www.openbet.com/> Michal Borowiecki
> Senior Software Engineer L4
> T: +44 208 742 1600
>
>
> +44 203 249 8448
>
>
>
> E: michal.borowiecki@openbet.com
> W: www.openbet.com <http://www.openbet.com/>
>
>
> OpenBet Ltd
>
> Chiswick Park Building 9
>
> 566 Chiswick High Rd
>
> London
>
> W4 5XT
>
> UK
>
>
> <https://www.openbet.com/email_promo>
>
> This message is confidential and intended only for the addressee. If you
> have received this message in error, please immediately notify the
> postmaster@openbet.com <ma...@openbet.com> and delete it
> from your system as well as any copies. The content of e-mails as well
> as traffic data may be monitored by OpenBet for employment and security
> purposes. To protect the environment please do not print this e-mail
> unless necessary. OpenBet Ltd. Registered Office: Chiswick Park Building
> 9, 566 Chiswick High Road, London, W4 5XT, United Kingdom. A company
> registered in England and Wales. Registered no. 3134634. VAT no.
> GB927523612
>
Re: Sink Processor definition
Posted by Michal Borowiecki <mi...@openbet.com>.
Yes, I think the key distinction, from the point of view of that
documentation section, is that it doesn't have downstream processors.
On 03/06/17 09:48, Damian Guy wrote:
> Hi Michal,
>
> In this case Sink Processor is really referring to a SinkNode that can
> only produce to a kafka topic. Maybe the terminology is incorrect as
> strictly speaking a processor that writes data to anything could be
> considered a Sink Processor.
>
> On Sat, 3 Jun 2017 at 09:23 Michal Borowiecki
> <michal.borowiecki@openbet.com <ma...@openbet.com>>
> wrote:
>
> Hi all,
>
> Streams docs say:
>
>> * *Sink Processor*: A sink processor is a special type of
>> stream processor that does not have down-stream processors.
>> It sends any received records from its up-stream processors
>> to a specified Kafka topic.
>>
> Would a processor that doesn't produce to a kafka topic (directly)
> but only updates a state store also be considered a sink
> processor? I think yes.
>
> I'll submit a PR to that effect unless I hear otherwise.
>
> Cheers,
>
> Michał
>
> --
> <http://www.openbet.com/> Michal Borowiecki
> Senior Software Engineer L4
> T: +44 208 742 1600 <tel:+44%2020%208742%201600>
>
>
> +44 203 249 8448 <tel:+44%2020%203249%208448>
>
>
>
> E: michal.borowiecki@openbet.com
> <ma...@openbet.com>
> W: www.openbet.com <http://www.openbet.com/>
>
>
> OpenBet Ltd
>
> Chiswick Park Building 9
>
> 566 Chiswick High Rd
>
> London
>
> W4 5XT
>
> UK
>
>
> <https://www.openbet.com/email_promo>
>
> This message is confidential and intended only for the addressee.
> If you have received this message in error, please immediately
> notify the postmaster@openbet.com <ma...@openbet.com>
> and delete it from your system as well as any copies. The content
> of e-mails as well as traffic data may be monitored by OpenBet for
> employment and security purposes. To protect the environment
> please do not print this e-mail unless necessary. OpenBet Ltd.
> Registered Office: Chiswick Park Building 9, 566 Chiswick High
> Road, London, W4 5XT, United Kingdom. A company registered in
> England and Wales. Registered no. 3134634. VAT no. GB927523612
>
--
Signature
<http://www.openbet.com/> Michal Borowiecki
Senior Software Engineer L4
T: +44 208 742 1600
+44 203 249 8448
E: michal.borowiecki@openbet.com
W: www.openbet.com <http://www.openbet.com/>
OpenBet Ltd
Chiswick Park Building 9
566 Chiswick High Rd
London
W4 5XT
UK
<https://www.openbet.com/email_promo>
This message is confidential and intended only for the addressee. If you
have received this message in error, please immediately notify the
postmaster@openbet.com <ma...@openbet.com> and delete it
from your system as well as any copies. The content of e-mails as well
as traffic data may be monitored by OpenBet for employment and security
purposes. To protect the environment please do not print this e-mail
unless necessary. OpenBet Ltd. Registered Office: Chiswick Park Building
9, 566 Chiswick High Road, London, W4 5XT, United Kingdom. A company
registered in England and Wales. Registered no. 3134634. VAT no.
GB927523612
Re: Sink Processor definition
Posted by Damian Guy <da...@gmail.com>.
Hi Michal,
In this case Sink Processor is really referring to a SinkNode that can only
produce to a kafka topic. Maybe the terminology is incorrect as strictly
speaking a processor that writes data to anything could be considered a
Sink Processor.
On Sat, 3 Jun 2017 at 09:23 Michal Borowiecki <mi...@openbet.com>
wrote:
> Hi all,
>
> Streams docs say:
>
>
> - *Sink Processor*: A sink processor is a special type of stream
> processor that does not have down-stream processors. It sends any received
> records from its up-stream processors to a specified Kafka topic.
>
> Would a processor that doesn't produce to a kafka topic (directly) but
> only updates a state store also be considered a sink processor? I think yes.
>
> I'll submit a PR to that effect unless I hear otherwise.
>
> Cheers,
>
> Michał
> --
> <http://www.openbet.com/> Michal Borowiecki
> Senior Software Engineer L4
> T: +44 208 742 1600 <+44%2020%208742%201600>
>
>
> +44 203 249 8448 <+44%2020%203249%208448>
>
>
>
> E: michal.borowiecki@openbet.com
> W: www.openbet.com
> OpenBet Ltd
>
> Chiswick Park Building 9
>
> 566 Chiswick High Rd
>
> London
>
> W4 5XT
>
> UK
> <https://www.openbet.com/email_promo>
> This message is confidential and intended only for the addressee. If you
> have received this message in error, please immediately notify the
> postmaster@openbet.com and delete it from your system as well as any
> copies. The content of e-mails as well as traffic data may be monitored by
> OpenBet for employment and security purposes. To protect the environment
> please do not print this e-mail unless necessary. OpenBet Ltd. Registered
> Office: Chiswick Park Building 9, 566 Chiswick High Road, London, W4 5XT,
> United Kingdom. A company registered in England and Wales. Registered no.
> 3134634. VAT no. GB927523612
>