Building news feed of social app using kafka

[Anshuman Ghosh <an...@terriblytinytales.com>]

Hello!

I am currently designing a social app (with the whole gamut of users
following each other and personal feeds - consisting of posts by those you
follow). To implement this "news feed" for each user, I was considering
having a Kafka stream/topic per user.

Given that our intention is to get 1M+ users on the app, is this a good
idea to have 1 topic per user, thus ending up with a million topics?

Thanks and regards,
Anshuman

Re: Building news feed of social app using kafka

[Svante Karlsson <sv...@csi.se>]

Nope, that's the wrong design. It does not scale. You would end up in a
wide and shallow thing. To few messages per partition to make sense. You
want many thousands per partition per second to amortize the consumer to
broker round-trip.


On Nov 1, 2017 21:12, "Anshuman Ghosh" <an...@terriblytinytales.com>
wrote:

> Hello!
>
> I am currently designing a social app (with the whole gamut of users
> following each other and personal feeds - consisting of posts by those you
> follow). To implement this "news feed" for each user, I was considering
> having a Kafka stream/topic per user.
>
> Given that our intention is to get 1M+ users on the app, is this a good
> idea to have 1 topic per user, thus ending up with a million topics?
>
> Thanks and regards,
> Anshuman
>