You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Fares Oueslati <ou...@gmail.com> on 2020/04/06 14:19:39 UTC
Topics naming convention
Hello,
I'm here to get some advice regarding Kafka topics naming.
If I name my topic "users" and I publish Avro events with a schema registry
using a given compatibility mode.
One day, due to business reasons, I have to break my schema, what would you
do?
add a new topic (let's call it "users-v1") ?
add a new schema registry subject and link it to the same topic "users"?
I would go for the first one (naming my topic users-v1 at first and then
incrementing) but I was wondering about consumers side effects if one day
we decide to change the topic (how they will handle offsets etc.) That's
why I m asking :)
Thank you guys!
Re: [External] Topics naming convention
Posted by Brian Sang <ba...@yelp.com.INVALID>.
If the schema is no longer compatible I think it makes sense to create a
new topic.
Offsets wise you can just have the consumer finish consuming from the old
topic/schema (i.e. migrate all producers to use the new schema format
first) and then migrate consumers to use the new schema format (and thus
new topic). Alternatively you can run separate instances of the consumer,
some consuming from old topic and some consuming from new.
On Mon, Apr 6, 2020 at 7:20 AM Fares Oueslati <ou...@gmail.com>
wrote:
> Hello,
>
> I'm here to get some advice regarding Kafka topics naming.
>
> If I name my topic "users" and I publish Avro events with a schema registry
> using a given compatibility mode.
> One day, due to business reasons, I have to break my schema, what would you
> do?
>
> add a new topic (let's call it "users-v1") ?
> add a new schema registry subject and link it to the same topic "users"?
>
> I would go for the first one (naming my topic users-v1 at first and then
> incrementing) but I was wondering about consumers side effects if one day
> we decide to change the topic (how they will handle offsets etc.) That's
> why I m asking :)
>
> Thank you guys!
>
Re: [External] Topics naming convention
Posted by Brian Sang <ba...@yelp.com.INVALID>.
If the schema is no longer compatible I think it makes sense to create a
new topic.
Offsets wise you can just have the consumer finish consuming from the old
topic/schema (i.e. migrate all producers to use the new schema format
first) and then migrate consumers to use the new schema format (and thus
new topic). Alternatively you can run separate instances of the consumer,
some consuming from old topic and some consuming from new.
On Mon, Apr 6, 2020 at 7:20 AM Fares Oueslati <ou...@gmail.com>
wrote:
> Hello,
>
> I'm here to get some advice regarding Kafka topics naming.
>
> If I name my topic "users" and I publish Avro events with a schema registry
> using a given compatibility mode.
> One day, due to business reasons, I have to break my schema, what would you
> do?
>
> add a new topic (let's call it "users-v1") ?
> add a new schema registry subject and link it to the same topic "users"?
>
> I would go for the first one (naming my topic users-v1 at first and then
> incrementing) but I was wondering about consumers side effects if one day
> we decide to change the topic (how they will handle offsets etc.) That's
> why I m asking :)
>
> Thank you guys!
>