You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Vahid S Hashemian <va...@us.ibm.com> on 2016/07/26 00:20:03 UTC
Potential Change in Kafka's Partition Assignment Semantics when
Subscription Changes
Hello,
We have started a KIP under the Kafka project that proposes a fix for an
inconsistency in how partition assignments are currently handled in Kafka
when the consumer changes subscription. Note that this applies to new
consumer only.
The KIP can be found here:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-70%3A+Revise+Partition+Assignment+Semantics+on+New+Consumer%27s+Subscription+Change
The compatibility section (
https://cwiki.apache.org/confluence/display/KAFKA/KIP-70%3A+Revise+Partition+Assignment+Semantics+on+New+Consumer%27s+Subscription+Change#KIP-70:RevisePartitionAssignmentSemanticsonNewConsumer'sSubscriptionChange-Compatibility,Deprecation,andMigrationPlan
) describes impacted users.
We would like to know if any Apache Storm user would be affected by this
change. Thanks.
Regards,
--Vahid
Re: Potential Change in Kafka's Partition Assignment Semantics when
Subscription Changes
Posted by Cody Koeninger <co...@koeninger.org>.
This seems really low risk to me. In order to be impacted, it'd have
to be someone who was using the kafka integration in spark 2.0, which
isn't even officially released yet.
On Mon, Jul 25, 2016 at 7:23 PM, Vahid S Hashemian
<va...@us.ibm.com> wrote:
> Sorry, meant to ask if any Apache Sparkuser would be affected.
>
> --Vahid
>
>
>
> From: Vahid S Hashemian/Silicon Valley/IBM@IBMUS
> To: user@spark.apache.org, dev@spark.apache.org
> Date: 07/25/2016 05:21 PM
> Subject: Potential Change in Kafka's Partition Assignment Semantics
> when Subscription Changes
> ________________________________
>
>
>
> Hello,
>
> We have started a KIP under the Kafka project that proposes a fix for an
> inconsistency in how partition assignments are currently handled in Kafka
> when the consumer changes subscription. Note that this applies to new
> consumer only.
> The KIP can be found here:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-70%3A+Revise+Partition+Assignment+Semantics+on+New+Consumer%27s+Subscription+Change
>
> The compatibility section
> (https://cwiki.apache.org/confluence/display/KAFKA/KIP-70%3A+Revise+Partition+Assignment+Semantics+on+New+Consumer%27s+Subscription+Change#KIP-70:RevisePartitionAssignmentSemanticsonNewConsumer'sSubscriptionChange-Compatibility,Deprecation,andMigrationPlan)
> describes impacted users.
>
> We would like to know if any Apache Storm user would be affected by this
> change. Thanks.
>
> Regards,
> --Vahid
>
>
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
Re: Potential Change in Kafka's Partition Assignment Semantics when
Subscription Changes
Posted by Cody Koeninger <co...@koeninger.org>.
This seems really low risk to me. In order to be impacted, it'd have
to be someone who was using the kafka integration in spark 2.0, which
isn't even officially released yet.
On Mon, Jul 25, 2016 at 7:23 PM, Vahid S Hashemian
<va...@us.ibm.com> wrote:
> Sorry, meant to ask if any Apache Sparkuser would be affected.
>
> --Vahid
>
>
>
> From: Vahid S Hashemian/Silicon Valley/IBM@IBMUS
> To: user@spark.apache.org, dev@spark.apache.org
> Date: 07/25/2016 05:21 PM
> Subject: Potential Change in Kafka's Partition Assignment Semantics
> when Subscription Changes
> ________________________________
>
>
>
> Hello,
>
> We have started a KIP under the Kafka project that proposes a fix for an
> inconsistency in how partition assignments are currently handled in Kafka
> when the consumer changes subscription. Note that this applies to new
> consumer only.
> The KIP can be found here:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-70%3A+Revise+Partition+Assignment+Semantics+on+New+Consumer%27s+Subscription+Change
>
> The compatibility section
> (https://cwiki.apache.org/confluence/display/KAFKA/KIP-70%3A+Revise+Partition+Assignment+Semantics+on+New+Consumer%27s+Subscription+Change#KIP-70:RevisePartitionAssignmentSemanticsonNewConsumer'sSubscriptionChange-Compatibility,Deprecation,andMigrationPlan)
> describes impacted users.
>
> We would like to know if any Apache Storm user would be affected by this
> change. Thanks.
>
> Regards,
> --Vahid
>
>
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org
Re: Potential Change in Kafka's Partition Assignment Semantics when
Subscription Changes
Posted by Vahid S Hashemian <va...@us.ibm.com>.
Sorry, meant to ask if any Apache Spark user would be affected.
--Vahid
From: Vahid S Hashemian/Silicon Valley/IBM@IBMUS
To: user@spark.apache.org, dev@spark.apache.org
Date: 07/25/2016 05:21 PM
Subject: Potential Change in Kafka's Partition Assignment Semantics
when Subscription Changes
Hello,
We have started a KIP under the Kafka project that proposes a fix for an
inconsistency in how partition assignments are currently handled in Kafka
when the consumer changes subscription. Note that this applies to new
consumer only.
The KIP can be found here:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-70%3A+Revise+Partition+Assignment+Semantics+on+New+Consumer%27s+Subscription+Change
The compatibility section (
https://cwiki.apache.org/confluence/display/KAFKA/KIP-70%3A+Revise+Partition+Assignment+Semantics+on+New+Consumer%27s+Subscription+Change#KIP-70:RevisePartitionAssignmentSemanticsonNewConsumer'sSubscriptionChange-Compatibility,Deprecation,andMigrationPlan
) describes impacted users.
We would like to know if any Apache Storm user would be affected by this
change. Thanks.
Regards,
--Vahid