You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@kafka.apache.org by Denis Mikhaylov <no...@gmail.com> on 2016/07/19 07:48:12 UTC

Topic naming convention and common message envelope.

Hi, I plan to use Kafka for event-based integration between services. And I have two questions that bother me a lot:

1) What topic naming convention do you use?

2) Do you wrap your messages in some kind of envelope that contains metadata like originating system info, timestamp, event type, unique event identifier, etc? 

Thanks,
Denis.

Re: Topic naming convention and common message envelope.

Posted by Denis Mikhaylov <no...@gmail.com>.

Thanks Ewen! Do you have an example of your namings and maybe how your envelope looks? Just curious.

> On 24 июля 2016 г., at 3:41, Ewen Cheslack-Postava <ew...@confluent.io> wrote:
> 
> On Tue, Jul 19, 2016 at 12:48 AM, Denis Mikhaylov <no...@gmail.com>
> wrote:
> 
>> Hi, I plan to use Kafka for event-based integration between services. And
>> I have two questions that bother me a lot:
>> 
>> 1) What topic naming convention do you use?
> 
> There's no strict convention, but using '.' or '_' to indicate hierarchical
> organization helps (and may eventually map to built-in hierarchical topics).
> 
> 
>> 
>> 2) Do you wrap your messages in some kind of envelope that contains
>> metadata like originating system info, timestamp, event type, unique event
>> identifier, etc?
> 
> This is a good practice, and something we've thought about adding support
> for in Confluent's schema registry. Just make sure it is *strongly*
> backwards compatible, i.e. only add fields w/ default values. Having a
> common header/envelope gives you a lot of useful info across all your
> topics.
> 
> -Ewen
> 
> 
>> 
>> Thanks,
>> Denis.
> 
> 
> 
> 
> -- 
> Thanks,
> Ewen

Re: Topic naming convention and common message envelope.

Posted by Ewen Cheslack-Postava <ew...@confluent.io>.

On Tue, Jul 19, 2016 at 12:48 AM, Denis Mikhaylov <no...@gmail.com>
wrote:

> Hi, I plan to use Kafka for event-based integration between services. And
> I have two questions that bother me a lot:
>
> 1) What topic naming convention do you use?
>

There's no strict convention, but using '.' or '_' to indicate hierarchical
organization helps (and may eventually map to built-in hierarchical topics).

>
> 2) Do you wrap your messages in some kind of envelope that contains
> metadata like originating system info, timestamp, event type, unique event
> identifier, etc?
>

This is a good practice, and something we've thought about adding support
for in Confluent's schema registry. Just make sure it is *strongly*
backwards compatible, i.e. only add fields w/ default values. Having a
common header/envelope gives you a lot of useful info across all your
topics.

-Ewen

>
> Thanks,
> Denis.

-- 
Thanks,
Ewen