You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@samza.apache.org by 舒琦 <sh...@eefung.com> on 2017/07/11 06:56:56 UTC
A Question About multi-output
Hi guys,
How to write one message to two different output stream, let’s say two different kafka cluster?
For what I understand now, MessageCollector.send can only be called once in process, because it relates to Acks, checkpoint and processing loop.
Thanks for you help!
————————
Qi Shu
Re: A Question About multi-output
Posted by 舒琦 <sh...@eefung.com>.
Hi Jagadish,
Thank you very much!
————————
QiShu
> 在 2017年7月12日，10:45，Jagadish Venkatraman <ja...@gmail.com> 写道：
>
> Yes, before the completion of a checkpoint/commit, all pending messages
> (across all streams) are guaranteed to be flushed.
>
> On Tue, Jul 11, 2017 at 7:33 PM 舒琦 <sh...@eefung.com> wrote:
>
>> Hi Jagadish,
>>
>> When write one message to two different output streams, Samza will
>> wait acks from two different streams and then process checkpointing. Do I
>> understand right?
>>
>> Thanks for your help!
>>
>> ————————
>> QiShu
>>
>>> 在 2017年7月12日，03:48，Jagadish Venkatraman <ja...@gmail.com> 写道：
>>>
>>> To write to two different output streams, you can call collector.send()
>>> twice.
>>>
>>>>> For what I understand now, MessageCollector.send can only be
>>> called once in process, because it relates to Acks, checkpoint and
>>> processing loop.
>>>
>>> These should be pretty orthogonal. If the docs are misleading, please let
>>> us know. We'll fix them
>>>
>>> On Mon, Jul 10, 2017 at 11:56 PM, 舒琦 <sh...@eefung.com> wrote:
>>>
>>>> Hi guys,
>>>>
>>>> How to write one message to two different output stream, let’s
>>>> say two different kafka cluster?
>>>>
>>>> For what I understand now, MessageCollector.send can only be
>>>> called once in process, because it relates to Acks, checkpoint and
>>>> processing loop.
>>>>
>>>> Thanks for you help!
>>>>
>>>> ————————
>>>> Qi Shu
>>>>
>>>>
>>>
>>>
>>> --
>>> Jagadish V,
>>> Graduate Student,
>>> Department of Computer Science,
>>> Stanford University
>>
>> --
> Sent from my iphone.
Re: A Question About multi-output
Posted by Jagadish Venkatraman <ja...@gmail.com>.
Yes, before the completion of a checkpoint/commit, all pending messages
(across all streams) are guaranteed to be flushed.
On Tue, Jul 11, 2017 at 7:33 PM 舒琦 <sh...@eefung.com> wrote:
> Hi Jagadish,
>
> When write one message to two different output streams, Samza will
> wait acks from two different streams and then process checkpointing. Do I
> understand right?
>
> Thanks for your help!
>
> ————————
> QiShu
>
> > 在 2017年7月12日，03:48，Jagadish Venkatraman <ja...@gmail.com> 写道：
> >
> > To write to two different output streams, you can call collector.send()
> > twice.
> >
> >>> For what I understand now, MessageCollector.send can only be
> > called once in process, because it relates to Acks, checkpoint and
> > processing loop.
> >
> > These should be pretty orthogonal. If the docs are misleading, please let
> > us know. We'll fix them
> >
> > On Mon, Jul 10, 2017 at 11:56 PM, 舒琦 <sh...@eefung.com> wrote:
> >
> >> Hi guys,
> >>
> >> How to write one message to two different output stream, let’s
> >> say two different kafka cluster?
> >>
> >> For what I understand now, MessageCollector.send can only be
> >> called once in process, because it relates to Acks, checkpoint and
> >> processing loop.
> >>
> >> Thanks for you help!
> >>
> >> ————————
> >> Qi Shu
> >>
> >>
> >
> >
> > --
> > Jagadish V,
> > Graduate Student,
> > Department of Computer Science,
> > Stanford University
>
> --
Sent from my iphone.
Re: A Question About multi-output
Posted by 舒琦 <sh...@eefung.com>.
Hi Jagadish,
When write one message to two different output streams, Samza will wait acks from two different streams and then process checkpointing. Do I understand right?
Thanks for your help!
————————
QiShu
> 在 2017年7月12日，03:48，Jagadish Venkatraman <ja...@gmail.com> 写道：
>
> To write to two different output streams, you can call collector.send()
> twice.
>
>>> For what I understand now, MessageCollector.send can only be
> called once in process, because it relates to Acks, checkpoint and
> processing loop.
>
> These should be pretty orthogonal. If the docs are misleading, please let
> us know. We'll fix them
>
> On Mon, Jul 10, 2017 at 11:56 PM, 舒琦 <sh...@eefung.com> wrote:
>
>> Hi guys,
>>
>> How to write one message to two different output stream, let’s
>> say two different kafka cluster?
>>
>> For what I understand now, MessageCollector.send can only be
>> called once in process, because it relates to Acks, checkpoint and
>> processing loop.
>>
>> Thanks for you help!
>>
>> ————————
>> Qi Shu
>>
>>
>
>
> --
> Jagadish V,
> Graduate Student,
> Department of Computer Science,
> Stanford University
Re: A Question About multi-output
Posted by Jagadish Venkatraman <ja...@gmail.com>.
To write to two different output streams, you can call collector.send()
twice.
>> For what I understand now, MessageCollector.send can only be
called once in process, because it relates to Acks, checkpoint and
processing loop.
These should be pretty orthogonal. If the docs are misleading, please let
us know. We'll fix them
On Mon, Jul 10, 2017 at 11:56 PM, 舒琦 <sh...@eefung.com> wrote:
> Hi guys,
>
> How to write one message to two different output stream, let’s
> say two different kafka cluster?
>
> For what I understand now, MessageCollector.send can only be
> called once in process, because it relates to Acks, checkpoint and
> processing loop.
>
> Thanks for you help!
>
> ————————
> Qi Shu
>
>
--
Jagadish V,
Graduate Student,
Department of Computer Science,
Stanford University