You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Kenneth Knowles <ke...@apache.org> on 2024/08/15 17:46:31 UTC

[PROPOSAL] 2.8.1 point release due to potential data duplication in KafkaIO

Hi all

I am proposing a point release because 2.8.0 has a critical data bug. The
KafkaIO unconditionally adds a "redistribute allowing duplicates" transform
to the read path, whether or not a user requested it.

Issue: https://github.com/apache/beam/issues/32196
Fix: https://github.com/apache/beam/pull/32134

Luckily most runners do not support "allowing duplicates" yet so the impact
is minimal. However, let's release an SDK without such a surprise lurking
in it.

Kenn

Re: [PROPOSAL] 2.8.1 point release due to potential data duplication in KafkaIO

Posted by Kenneth Knowles via dev <de...@beam.apache.org>.
Aha yes 2.58.0 and 2.58.1, respectively. I'm not proposing that we create a
release at 88 miles per hour.

Kenn

On Thu, Aug 15, 2024 at 2:10 PM Jack McCluskey via dev <de...@beam.apache.org>
wrote:

> Hey everyone,
>
> I'm assuming you mean 2.58.1? I'm +1 on a patch release to fix the problem
> though.
>
> Thanks,
>
> Jack McCluskey
>
> On Thu, Aug 15, 2024 at 1:48 PM Kenneth Knowles <ke...@apache.org> wrote:
>
>> Hi all
>>
>> I am proposing a point release because 2.8.0 has a critical data bug. The
>> KafkaIO unconditionally adds a "redistribute allowing duplicates" transform
>> to the read path, whether or not a user requested it.
>>
>> Issue: https://github.com/apache/beam/issues/32196
>> Fix: https://github.com/apache/beam/pull/32134
>>
>> Luckily most runners do not support "allowing duplicates" yet so the
>> impact is minimal. However, let's release an SDK without such a surprise
>> lurking in it.
>>
>> Kenn
>>
>

Re: [PROPOSAL] 2.8.1 point release due to potential data duplication in KafkaIO

Posted by Danny McCormick via dev <de...@beam.apache.org>.
The candidate is fully complete and ready for validation! Please take a
look. Because of the urgent nature of the fix and the small changeset
involved (a single commit), I am proposing that we accelerate the normal
release timeline and try to get this out before the weekend. We can move
that conversation to the actual RC voting thread, though. Thread -
https://lists.apache.org/thread/blxjd8v3gyfdt8cmkp9z3vhh4rwqn038

Thanks,
Danny

On Thu, Aug 15, 2024 at 8:23 PM Danny McCormick <da...@google.com>
wrote:

> +1 on doing a patch release. I've pushed a release branch and am building
> a candidate, I'll follow up once I have a candidate built/ready to go, and
> we can proceed with that unless there are objections.
>
> Thanks,
> Danny
>
> On Thu, Aug 15, 2024 at 8:10 PM Jack McCluskey via dev <
> dev@beam.apache.org> wrote:
>
>> Hey everyone,
>>
>> I'm assuming you mean 2.58.1? I'm +1 on a patch release to fix the
>> problem though.
>>
>> Thanks,
>>
>> Jack McCluskey
>>
>> On Thu, Aug 15, 2024 at 1:48 PM Kenneth Knowles <ke...@apache.org> wrote:
>>
>>> Hi all
>>>
>>> I am proposing a point release because 2.8.0 has a critical data bug.
>>> The KafkaIO unconditionally adds a "redistribute allowing duplicates"
>>> transform to the read path, whether or not a user requested it.
>>>
>>> Issue: https://github.com/apache/beam/issues/32196
>>> Fix: https://github.com/apache/beam/pull/32134
>>>
>>> Luckily most runners do not support "allowing duplicates" yet so the
>>> impact is minimal. However, let's release an SDK without such a surprise
>>> lurking in it.
>>>
>>> Kenn
>>>
>>

Re: [PROPOSAL] 2.8.1 point release due to potential data duplication in KafkaIO

Posted by Danny McCormick via dev <de...@beam.apache.org>.
+1 on doing a patch release. I've pushed a release branch and am building a
candidate, I'll follow up once I have a candidate built/ready to go, and we
can proceed with that unless there are objections.

Thanks,
Danny

On Thu, Aug 15, 2024 at 8:10 PM Jack McCluskey via dev <de...@beam.apache.org>
wrote:

> Hey everyone,
>
> I'm assuming you mean 2.58.1? I'm +1 on a patch release to fix the problem
> though.
>
> Thanks,
>
> Jack McCluskey
>
> On Thu, Aug 15, 2024 at 1:48 PM Kenneth Knowles <ke...@apache.org> wrote:
>
>> Hi all
>>
>> I am proposing a point release because 2.8.0 has a critical data bug. The
>> KafkaIO unconditionally adds a "redistribute allowing duplicates" transform
>> to the read path, whether or not a user requested it.
>>
>> Issue: https://github.com/apache/beam/issues/32196
>> Fix: https://github.com/apache/beam/pull/32134
>>
>> Luckily most runners do not support "allowing duplicates" yet so the
>> impact is minimal. However, let's release an SDK without such a surprise
>> lurking in it.
>>
>> Kenn
>>
>

Re: [PROPOSAL] 2.8.1 point release due to potential data duplication in KafkaIO

Posted by Jack McCluskey via dev <de...@beam.apache.org>.
Hey everyone,

I'm assuming you mean 2.58.1? I'm +1 on a patch release to fix the problem
though.

Thanks,

Jack McCluskey

On Thu, Aug 15, 2024 at 1:48 PM Kenneth Knowles <ke...@apache.org> wrote:

> Hi all
>
> I am proposing a point release because 2.8.0 has a critical data bug. The
> KafkaIO unconditionally adds a "redistribute allowing duplicates" transform
> to the read path, whether or not a user requested it.
>
> Issue: https://github.com/apache/beam/issues/32196
> Fix: https://github.com/apache/beam/pull/32134
>
> Luckily most runners do not support "allowing duplicates" yet so the
> impact is minimal. However, let's release an SDK without such a surprise
> lurking in it.
>
> Kenn
>