You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by shyla deshpande <de...@gmail.com> on 2017/02/26 07:10:53 UTC

In Spark streaming, will saved kafka offsets become invalid if I change the number of partitions in a kafka topic?

I am commiting offsets to Kafka after my output has been stored, using the
commitAsync API.

My question is if I increase/decrease the number of kafka partitions, will
the saved offsets will become invalid.

Thanks

Re: In Spark streaming, will saved kafka offsets become invalid if I change the number of partitions in a kafka topic?

Posted by shyla deshpande <de...@gmail.com>.

Please help!

On Sat, Feb 25, 2017 at 11:10 PM, shyla deshpande <de...@gmail.com>
wrote:

> I am commiting offsets to Kafka after my output has been stored, using the
> commitAsync API.
>
> My question is if I increase/decrease the number of kafka partitions, will
> the saved offsets will become invalid.
>
> Thanks
>