Re: [DISCUSS] KIP-349 Priorities for Source Topics

[Jan Filipiak <Ja...@trivago.com>]

On 30.08.2018 15:17, Matthias J. Sax wrote:
> Nick,
>
> Would be good to understand the difference between the current approach
> and what Jan suggested. If we believe that the current proposal is too
> limited in functionality and also hard to extend later on, it might make
> sense to work on a more generic solution from the beginning on. On the
> other hand, if we can extend the current proposal easily, I see no
> reason to build this incrementally.
Difference is that from my POV topic priorities is a special 
implementation of a MessageChooser

>
> @Jan: Can you summarize the differences from you point of view? You also
> linked to KIP-353 in the VOTE thread -- how does it related to this KIP?
The relationship is rather obvious to me. KIP-353 can be implemented as 
a MessageChooser
>
> -Matthias
To add more confusion, why not also make "pause partition" as in connect 
part of the MessageChooser interface?

>
>
> On 8/12/18 11:15 PM, Matt Farmer wrote:
>> Ah, sorry, yes it does.
>>
>> On Sun, Aug 12, 2018 at 2:58 PM <ni...@afshartous.com> wrote:
>>
>>> Does this clarify ?
>>> --
>>>        Nick
>>>
>>> On Aug 9, 2018, at 7:44 PM, nick@afshartous.com wrote:
>>>
>>> Since there are questions I changed the heading from VOTE to DISCUSS
>>>
>>> On Aug 8, 2018, at 9:09 PM, Matt Farmer <ma...@frmr.me> wrote:
>>>
>>> s it worth spelling out explicitly what the behavior is when two topics
>>> have the same priority? I'm a bit fuzzy on how we choose what topics to
>>> consume from right now, if I'm being honest, so it could be useful to
>>> outline the current behavior in the background and to spell out how that
>>> would change (or if it would change) when two topics are given the same
>>> priority.
>>>
>>>
>>> I added an additional note in the KIP’s Compatibility section to clarify
>>> that current behavior would not change in order to preserve backwards
>>> compatibility.
>>>
>>> Also, how does this play with max.poll.records? Does the consumer read from
>>>
>>> all the topics in priority order until we've hit the number of records or
>>> the poll timeout? Or does it immediately return the high priority records
>>> without pulling low priority records?
>>>
>>>
>>> My own interpretation would be to read from all the topics in priority
>>> order as the consumer is subscribed to multiple topics.
>>> --
>>>        Nick
>>>
>>>
>>>
>>>
>>>
>>>
>>>