Single consumer subscribing to multiple topics

[Manu Jacob <Ma...@sas.com>]

Hi Experts,

I have a requirement to consume records from multiple topics and process those messages with a similar pattern. The schema of the messages in each topic is different and I am consuming it as Generic records. Both these topics have very high rate of traffic (10-15K messages/second) and would have 50 partitions each (with partition key being pretty much distributed). The offsets are committed manually for these topics to keep it in sync with the processing checkpoints.  I was wondering whether I should use a single consumer subscribing to multiple topics or have separate consumer for each topic. Are there any considerable performance implications while consuming from multiple topics with a single consumer? I could not find any guidelines or much documentation on this. Would appreciate any guidelines on this.

Thanks,
-Manu