You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Aljoscha Krettek <al...@apache.org> on 2017/04/03 14:39:35 UTC
Re: [VOTE] Release Apache Flink 1.2.1 (RC1)
Just so we’re all on the same page. ;-)
There was https://issues.apache.org/jira/browse/FLINK-5808 which was a bug that we initially discovered in Flink 1.2 which was/is about missing verification for the correctness of the combination of parallelism and max-parallelism. Due to lacking test coverage this introduced two more bugs:
- https://issues.apache.org/jira/browse/FLINK-6188: Some setParallelism() methods can't cope with default parallelism
- https://issues.apache.org/jira/browse/FLINK-6209: StreamPlanEnvironment always has a parallelism of 1
IMHO, the options are:
1) revert the changes made for FLINK-5808 on the release-1.2 branch and live with the bug still being present
2) put in more work to fix FLINK-5808 which requires fixing some problems that have existed for a long time with how the parallelism is set in streaming programs
Best,
Aljoscha
> On 31. Mar 2017, at 21:34, Robert Metzger <rm...@apache.org> wrote:
>
> I don't know what is best to do, but I think releasing 1.2.1 with
> potentially more bugs than 1.2.0 is not a good option.
> I suspect a good workaround for FLINK-6188
> <https://issues.apache.org/jira/browse/FLINK-6188> is setting the
> parallelism manually for operators that can't cope with the default -1
> parallelism.
>
> On Fri, Mar 31, 2017 at 9:06 PM, Aljoscha Krettek <al...@apache.org>
> wrote:
>
>> You mean reverting the changes around FLINK-5808 [1]? This is what
>> introduced the follow-up FLINK-6188 [2].
>>
>> [1] https://issues.apache.org/jira/browse/FLINK-5808
>> [2]https://issues.apache.org/jira/browse/FLINK-6188
>>
>> On Fri, Mar 31, 2017, at 19:10, Robert Metzger wrote:
>>> I think reverting FLINK-6188 for the 1.2 branch might be a good idea.
>>> FLINK-6188 introduced two new bugs, so undoing the FLINK-6188 fix will
>>> lead
>>> only to one known bug in 1.2.1, instead of an uncertain number of issues.
>>> So 1.2.1 is not going to be worse than 1.2.0
>>>
>>> The fix will hopefully make it into 1.2.2 then.
>>>
>>> Any other thoughts on this?
>>>
>>>
>>>
>>>
>>> On Fri, Mar 31, 2017 at 6:46 PM, Fabian Hueske <fh...@gmail.com>
>> wrote:
>>>
>>>> I merged the fix for FLINK-6044 to the release-1.2 and release-1.1
>> branch.
>>>>
>>>> 2017-03-31 15:02 GMT+02:00 Fabian Hueske <fh...@gmail.com>:
>>>>
>>>>> We should also backport the fix for FLINK-6044 to Flink 1.2.1.
>>>>>
>>>>> I'll take care of that.
>>>>>
>>>>> 2017-03-30 18:50 GMT+02:00 Aljoscha Krettek <al...@apache.org>:
>>>>>
>>>>>> https://issues.apache.org/jira/browse/FLINK-6188 turns out to be a
>> bit
>>>>>> more involved, see my comments on the PR:
>>>>>> https://github.com/apache/flink/pull/3616.
>>>>>>
>>>>>> As I said there, maybe we should revert the commits regarding
>>>>>> parallelism/max-parallelism changes and release and then fix it
>> later.
>>>>>>
>>>>>> On Wed, Mar 29, 2017, at 23:08, Aljoscha Krettek wrote:
>>>>>>> I commented on FLINK-6214: I think it's working as intended,
>> although
>>>> we
>>>>>>> could fix the javadoc/doc.
>>>>>>>
>>>>>>> On Wed, Mar 29, 2017, at 17:35, Timo Walther wrote:
>>>>>>>> A user reported that all tumbling and slinding window assigners
>>>>>> contain
>>>>>>>> a pretty obvious bug about offsets.
>>>>>>>>
>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6214
>>>>>>>>
>>>>>>>> I think we should also fix this for 1.2.1. What do you think?
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Timo
>>>>>>>>
>>>>>>>>
>>>>>>>> Am 29/03/17 um 11:30 schrieb Robert Metzger:
>>>>>>>>> Hi Haohui,
>>>>>>>>> I agree that we should fix the parallelism issue. Otherwise,
>> the
>>>>>> 1.2.1
>>>>>>>>> release would introduce a new bug.
>>>>>>>>>
>>>>>>>>> On Tue, Mar 28, 2017 at 11:59 PM, Haohui Mai <
>> ricetons@gmail.com>
>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> -1 (non-binding)
>>>>>>>>>>
>>>>>>>>>> We recently found out that all jobs submitted via UI will
>> have a
>>>>>>>>>> parallelism of 1, potentially due to FLINK-5808.
>>>>>>>>>>
>>>>>>>>>> Filed FLINK-6209 to track it.
>>>>>>>>>>
>>>>>>>>>> ~Haohui
>>>>>>>>>>
>>>>>>>>>> On Mon, Mar 27, 2017 at 2:59 AM Chesnay Schepler <
>>>>>> chesnay@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> If possible I would like to include FLINK-6183 & FLINK-6184
>> as
>>>>>> well.
>>>>>>>>>>>
>>>>>>>>>>> They fix 2 metric-related issues that could arise when a
>> Task is
>>>>>>>>>>> cancelled very early. (like, right away)
>>>>>>>>>>>
>>>>>>>>>>> FLINK-6183 fixes a memory leak where the TaskMetricGroup was
>>>>>> never closed
>>>>>>>>>>> FLINK-6184 fixes a NullPointerExceptions in the buffer
>> metrics
>>>>>>>>>>>
>>>>>>>>>>> PR here: https://github.com/apache/flink/pull/3611
>>>>>>>>>>>
>>>>>>>>>>> On 26.03.2017 12:35, Aljoscha Krettek wrote:
>>>>>>>>>>>> I opened a PR for FLINK-6188: https://github.com/apache/
>>>>>>>>>> flink/pull/3616
>>>>>>>>>>> <https://github.com/apache/flink/pull/3616>
>>>>>>>>>>>> This improves the previously very sparse test coverage for
>>>>>>>>>>> timestamp/watermark assigners and fixes the bug.
>>>>>>>>>>>>> On 25 Mar 2017, at 10:22, Ufuk Celebi <uc...@apache.org>
>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> I agree with Aljoscha.
>>>>>>>>>>>>>
>>>>>>>>>>>>> -1 because of FLINK-6188
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Sat, Mar 25, 2017 at 9:38 AM, Aljoscha Krettek <
>>>>>>>>>> aljoscha@apache.org>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> I filed this issue, which was observed by a user:
>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6188
>>>>>>>>>>>>>> I think that’s blocking for 1.2.1.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On 24 Mar 2017, at 18:57, Ufuk Celebi <uc...@apache.org>
>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> RC1 doesn't contain Stefan's backport for the
>> Asynchronous
>>>>>> snapshots
>>>>>>>>>>>>>>> for heap-based keyed state that has been merged. Should
>> we
>>>>>> create
>>>>>>>>>> RC2
>>>>>>>>>>>>>>> with that fix since the voting period only starts on
>> Monday?
>>>>>> I think
>>>>>>>>>>>>>>> it would only mean rerunning the scripts on your side,
>>>> right?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> – Ufuk
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Fri, Mar 24, 2017 at 3:05 PM, Robert Metzger <
>>>>>>>>>> rmetzger@apache.org>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> Dear Flink community,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Please vote on releasing the following candidate as
>> Apache
>>>>>> Flink
>>>>>>>>>>> version 1.2
>>>>>>>>>>>>>>>> .1.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The commit to be voted on:
>>>>>>>>>>>>>>>> *732e55bd* (*
>>>>>>>>>>> http://git-wip-us.apache.org/repos/asf/flink/commit/
>> 732e55bd
>>>>>>>>>>>>>>>> <http://git-wip-us.apache.org/
>>>> repos/asf/flink/commit/732e55b
>>>>>> d>*)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Branch:
>>>>>>>>>>>>>>>> release-1.2.1-rc1
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The release artifacts to be voted on can be found at:
>>>>>>>>>>>>>>>> *http://people.apache.org/~rmetzger/flink-1.2.1-rc1/
>>>>>>>>>>>>>>>> <http://people.apache.org/~rmetzger/flink-1.2.1-rc1/>*
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The release artifacts are signed with the key with
>>>>>> fingerprint
>>>>>>>>>>> D9839159:
>>>>>>>>>>>>>>>> http://www.apache.org/dist/flink/KEYS
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The staging repository for this release can be found
>> at:
>>>>>>>>>>>>>>>>
>>>>>>>>>>> https://repository.apache.org/
>> content/repositories/orgapache
>>>>>> flink-1116
>>>>>>>>>>>>>>>> ------------------------------
>>>> ------------------------------
>>>>>> -
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The vote ends on Wednesday, March 29, 2017, 3pm CET.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> [ ] +1 Release this package as Apache Flink 1.2.1
>>>>>>>>>>>>>>>> [ ] -1 Do not release this package, because ...
>>>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>
Re: [VOTE] Release Apache Flink 1.2.1 (RC1)
Posted by Flavio Pompermaier <po...@okkam.it>.
Would it be possible to merge also the PR to fix FLINK-6103 (
https://github.com/apache/flink/pull/3598)....?
On Tue, Apr 4, 2017 at 12:03 PM, Chesnay Schepler <ch...@apache.org>
wrote:
> We can merge the metric changes; I'll rebase the branch and merge them
> within the next hours.
>
> On 04.04.2017 11:57, Robert Metzger wrote:
>
>> Thank you for opening a PR for this.
>>
>> Chesnay, do you need more reviews for the metrics changes / backports?
>>
>> Are there any other release blockers for 1.2.1, or are we good to go?
>>
>> On Mon, Apr 3, 2017 at 6:48 PM, Aljoscha Krettek <al...@apache.org>
>> wrote:
>>
>> I created a PR for the revert: https://github.com/apache/flink/pull/3664
>>>
>>> On 3. Apr 2017, at 18:32, Stephan Ewen <se...@apache.org> wrote:
>>>>
>>>> +1 for options (1), but also invest the time to fix it properly for
>>>> 1.2.2
>>>>
>>>>
>>>> On Mon, Apr 3, 2017 at 9:10 AM, Kostas Kloudas <
>>>>
>>> k.kloudas@data-artisans.com>
>>>
>>>> wrote:
>>>>
>>>> +1 for 1
>>>>>
>>>>> On Apr 3, 2017, at 5:52 PM, Till Rohrmann <tr...@apache.org>
>>>>>>
>>>>> wrote:
>>>
>>>> +1 for option 1)
>>>>>>
>>>>>> On Mon, Apr 3, 2017 at 5:48 PM, Fabian Hueske <fh...@gmail.com>
>>>>>>
>>>>> wrote:
>>>
>>>> +1 to option 1)
>>>>>>>
>>>>>>> 2017-04-03 16:57 GMT+02:00 Ted Yu <yu...@gmail.com>:
>>>>>>>
>>>>>>> Looks like #1 is better - 1.2.1 would be at least as stable as 1.2.0
>>>>>>>>
>>>>>>>> Cheers
>>>>>>>>
>>>>>>>> On Mon, Apr 3, 2017 at 7:39 AM, Aljoscha Krettek <
>>>>>>>>
>>>>>>> aljoscha@apache.org>
>>>
>>>> wrote:
>>>>>>>>
>>>>>>>> Just so we’re all on the same page. ;-)
>>>>>>>>>
>>>>>>>>> There was https://issues.apache.org/jira/browse/FLINK-5808 which
>>>>>>>>>
>>>>>>>> was
>>>
>>>> a
>>>>>
>>>>>> bug that we initially discovered in Flink 1.2 which was/is about
>>>>>>>>>
>>>>>>>> missing
>>>>>>>
>>>>>>>> verification for the correctness of the combination of parallelism
>>>>>>>>>
>>>>>>>> and
>>>
>>>> max-parallelism. Due to lacking test coverage this introduced two
>>>>>>>>>
>>>>>>>> more
>>>
>>>> bugs:
>>>>>>>>
>>>>>>>>> - https://issues.apache.org/jira/browse/FLINK-6188: Some
>>>>>>>>> setParallelism() methods can't cope with default parallelism
>>>>>>>>> - https://issues.apache.org/jira/browse/FLINK-6209:
>>>>>>>>> StreamPlanEnvironment always has a parallelism of 1
>>>>>>>>>
>>>>>>>>> IMHO, the options are:
>>>>>>>>> 1) revert the changes made for FLINK-5808 on the release-1.2 branch
>>>>>>>>>
>>>>>>>> and
>>>>>>>
>>>>>>>> live with the bug still being present
>>>>>>>>> 2) put in more work to fix FLINK-5808 which requires fixing some
>>>>>>>>>
>>>>>>>> problems
>>>>>>>>
>>>>>>>>> that have existed for a long time with how the parallelism is set
>>>>>>>>> in
>>>>>>>>> streaming programs
>>>>>>>>>
>>>>>>>>> Best,
>>>>>>>>> Aljoscha
>>>>>>>>>
>>>>>>>>> On 31. Mar 2017, at 21:34, Robert Metzger <rm...@apache.org>
>>>>>>>>>>
>>>>>>>>> wrote:
>>>>>>>
>>>>>>>> I don't know what is best to do, but I think releasing 1.2.1 with
>>>>>>>>>> potentially more bugs than 1.2.0 is not a good option.
>>>>>>>>>> I suspect a good workaround for FLINK-6188
>>>>>>>>>> <https://issues.apache.org/jira/browse/FLINK-6188> is setting the
>>>>>>>>>> parallelism manually for operators that can't cope with the
>>>>>>>>>> default
>>>>>>>>>>
>>>>>>>>> -1
>>>>>>>
>>>>>>>> parallelism.
>>>>>>>>>>
>>>>>>>>>> On Fri, Mar 31, 2017 at 9:06 PM, Aljoscha Krettek <
>>>>>>>>>>
>>>>>>>>> aljoscha@apache.org
>>>>>>>
>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> You mean reverting the changes around FLINK-5808 [1]? This is what
>>>>>>>>>>> introduced the follow-up FLINK-6188 [2].
>>>>>>>>>>>
>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/FLINK-5808
>>>>>>>>>>> [2]https://issues.apache.org/jira/browse/FLINK-6188
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Mar 31, 2017, at 19:10, Robert Metzger wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I think reverting FLINK-6188 for the 1.2 branch might be a good
>>>>>>>>>>>>
>>>>>>>>>>> idea.
>>>>>>>
>>>>>>>> FLINK-6188 introduced two new bugs, so undoing the FLINK-6188 fix
>>>>>>>>>>>>
>>>>>>>>>>> will
>>>>>>>>
>>>>>>>>> lead
>>>>>>>>>>>> only to one known bug in 1.2.1, instead of an uncertain number
>>>>>>>>>>>> of
>>>>>>>>>>>>
>>>>>>>>>>> issues.
>>>>>>>>>
>>>>>>>>>> So 1.2.1 is not going to be worse than 1.2.0
>>>>>>>>>>>>
>>>>>>>>>>>> The fix will hopefully make it into 1.2.2 then.
>>>>>>>>>>>>
>>>>>>>>>>>> Any other thoughts on this?
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Mar 31, 2017 at 6:46 PM, Fabian Hueske <
>>>>>>>>>>>>
>>>>>>>>>>> fhueske@gmail.com>
>>>
>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I merged the fix for FLINK-6044 to the release-1.2 and
>>>>>>>>>>>>>
>>>>>>>>>>>> release-1.1
>>>
>>>> branch.
>>>>>>>>>>>
>>>>>>>>>>>> 2017-03-31 15:02 GMT+02:00 Fabian Hueske <fh...@gmail.com>:
>>>>>>>>>>>>>
>>>>>>>>>>>>> We should also backport the fix for FLINK-6044 to Flink 1.2.1.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I'll take care of that.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2017-03-30 18:50 GMT+02:00 Aljoscha Krettek <
>>>>>>>>>>>>>>
>>>>>>>>>>>>> aljoscha@apache.org
>>>
>>>> :
>>>>>>>>
>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6188 turns out to
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> be
>>>>>>>
>>>>>>>> a
>>>>>>>>
>>>>>>>>> bit
>>>>>>>>>>>
>>>>>>>>>>>> more involved, see my comments on the PR:
>>>>>>>>>>>>>>> https://github.com/apache/flink/pull/3616.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> As I said there, maybe we should revert the commits regarding
>>>>>>>>>>>>>>> parallelism/max-parallelism changes and release and then fix
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> it
>>>
>>>> later.
>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Mar 29, 2017, at 23:08, Aljoscha Krettek wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I commented on FLINK-6214: I think it's working as intended,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> although
>>>>>>>>>>>
>>>>>>>>>>>> we
>>>>>>>>>>>>>
>>>>>>>>>>>>>> could fix the javadoc/doc.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Wed, Mar 29, 2017, at 17:35, Timo Walther wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> A user reported that all tumbling and slinding window
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> assigners
>>>>>>>
>>>>>>>> contain
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> a pretty obvious bug about offsets.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6214
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I think we should also fix this for 1.2.1. What do you
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> think?
>>>
>>>> Regards,
>>>>>>>>>>>>>>>>> Timo
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Am 29/03/17 um 11:30 schrieb Robert Metzger:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi Haohui,
>>>>>>>>>>>>>>>>>> I agree that we should fix the parallelism issue.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Otherwise,
>>>
>>>> the
>>>>>>>>>>>
>>>>>>>>>>>> 1.2.1
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> release would introduce a new bug.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Tue, Mar 28, 2017 at 11:59 PM, Haohui Mai <
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> ricetons@gmail.com>
>>>>>>>>>>>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> -1 (non-binding)
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> We recently found out that all jobs submitted via UI will
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> have a
>>>>>>>>>>>
>>>>>>>>>>>> parallelism of 1, potentially due to FLINK-5808.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Filed FLINK-6209 to track it.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> ~Haohui
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Mon, Mar 27, 2017 at 2:59 AM Chesnay Schepler <
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> chesnay@apache.org>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> If possible I would like to include FLINK-6183 &
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> FLINK-6184
>>>
>>>> as
>>>>>>>>>>>
>>>>>>>>>>>> well.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> They fix 2 metric-related issues that could arise when a
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Task is
>>>>>>>>>>>
>>>>>>>>>>>> cancelled very early. (like, right away)
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> FLINK-6183 fixes a memory leak where the TaskMetricGroup
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> was
>>>>>>>
>>>>>>>> never closed
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> FLINK-6184 fixes a NullPointerExceptions in the buffer
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> metrics
>>>>>>>>>>>
>>>>>>>>>>>> PR here: https://github.com/apache/flink/pull/3611
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On 26.03.2017 12:35, Aljoscha Krettek wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I opened a PR for FLINK-6188:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> https://github.com/apache/
>>>
>>>> flink/pull/3616
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> <https://github.com/apache/flink/pull/3616>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> This improves the previously very sparse test coverage
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> for
>>>
>>>> timestamp/watermark assigners and fixes the bug.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On 25 Mar 2017, at 10:22, Ufuk Celebi <uc...@apache.org>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I agree with Aljoscha.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> -1 because of FLINK-6188
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Sat, Mar 25, 2017 at 9:38 AM, Aljoscha Krettek <
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> aljoscha@apache.org>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I filed this issue, which was observed by a user:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6188
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I think that’s blocking for 1.2.1.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On 24 Mar 2017, at 18:57, Ufuk Celebi <
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> uce@apache.org>
>>>
>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> RC1 doesn't contain Stefan's backport for the
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Asynchronous
>>>>>>>>>>>
>>>>>>>>>>>> snapshots
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> for heap-based keyed state that has been merged.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Should
>>>
>>>> we
>>>>>>>>>>>
>>>>>>>>>>>> create
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> RC2
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> with that fix since the voting period only starts on
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Monday?
>>>>>>>>>>>
>>>>>>>>>>>> I think
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> it would only mean rerunning the scripts on your
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> side,
>>>
>>>> right?
>>>>>>>>>>>>>
>>>>>>>>>>>>>> – Ufuk
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Mar 24, 2017 at 3:05 PM, Robert Metzger <
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> rmetzger@apache.org>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Dear Flink community,
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Please vote on releasing the following candidate as
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Apache
>>>>>>>>>>>
>>>>>>>>>>>> Flink
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> version 1.2
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> .1.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> The commit to be voted on:
>>>>>>>>>>>>>>>>>>>>>>>>> *732e55bd* (*
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> http://git-wip-us.apache.org/r
>>>>>>>>>>>>>>>>>>>> epos/asf/flink/commit/
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 732e55bd
>>>>>>>>>>>
>>>>>>>>>>>> <http://git-wip-us.apache.org/
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> repos/asf/flink/commit/732e55b
>>>>>>>>>>>>>
>>>>>>>>>>>>>> d>*)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Branch:
>>>>>>>>>>>>>>>>>>>>>>>>> release-1.2.1-rc1
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> The release artifacts to be voted on can be found
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> at:
>>>
>>>> *http://people.apache.org/~
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> rmetzger/flink-1.2.1-rc1/
>>>
>>>> <http://people.apache.org/~
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> rmetzger/flink-1.2.1-rc1/
>>>
>>>> *
>>>>>>>>
>>>>>>>>> The release artifacts are signed with the key with
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> fingerprint
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> D9839159:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> http://www.apache.org/dist/flink/KEYS
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> The staging repository for this release can be
>>>>>>>>>>>>>>>>>>>>>>>>> found
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> at:
>>>>>>>>>>>
>>>>>>>>>>>> https://repository.apache.org/
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> content/repositories/orgapache
>>>>>>>>>>>
>>>>>>>>>>>> flink-1116
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>
>>>>>>>>>>>>>> -
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> The vote ends on Wednesday, March 29, 2017, 3pm
>>>>>>>>>>>>>>>>>>>>>>>>> CET.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> [ ] +1 Release this package as Apache Flink 1.2.1
>>>>>>>>>>>>>>>>>>>>>>>>> [ ] -1 Do not release this package, because ...
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>
>>>>>
>>>
>
--
Flavio Pompermaier
Development Department
OKKAM S.r.l.
Tel. +(39) 0461 1823908
Re: [VOTE] Release Apache Flink 1.2.1 (RC1)
Posted by Chesnay Schepler <ch...@apache.org>.
We can merge the metric changes; I'll rebase the branch and merge them
within the next hours.
On 04.04.2017 11:57, Robert Metzger wrote:
> Thank you for opening a PR for this.
>
> Chesnay, do you need more reviews for the metrics changes / backports?
>
> Are there any other release blockers for 1.2.1, or are we good to go?
>
> On Mon, Apr 3, 2017 at 6:48 PM, Aljoscha Krettek <al...@apache.org>
> wrote:
>
>> I created a PR for the revert: https://github.com/apache/flink/pull/3664
>>
>>> On 3. Apr 2017, at 18:32, Stephan Ewen <se...@apache.org> wrote:
>>>
>>> +1 for options (1), but also invest the time to fix it properly for 1.2.2
>>>
>>>
>>> On Mon, Apr 3, 2017 at 9:10 AM, Kostas Kloudas <
>> k.kloudas@data-artisans.com>
>>> wrote:
>>>
>>>> +1 for 1
>>>>
>>>>> On Apr 3, 2017, at 5:52 PM, Till Rohrmann <tr...@apache.org>
>> wrote:
>>>>> +1 for option 1)
>>>>>
>>>>> On Mon, Apr 3, 2017 at 5:48 PM, Fabian Hueske <fh...@gmail.com>
>> wrote:
>>>>>> +1 to option 1)
>>>>>>
>>>>>> 2017-04-03 16:57 GMT+02:00 Ted Yu <yu...@gmail.com>:
>>>>>>
>>>>>>> Looks like #1 is better - 1.2.1 would be at least as stable as 1.2.0
>>>>>>>
>>>>>>> Cheers
>>>>>>>
>>>>>>> On Mon, Apr 3, 2017 at 7:39 AM, Aljoscha Krettek <
>> aljoscha@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Just so we\u2019re all on the same page. ;-)
>>>>>>>>
>>>>>>>> There was https://issues.apache.org/jira/browse/FLINK-5808 which
>> was
>>>> a
>>>>>>>> bug that we initially discovered in Flink 1.2 which was/is about
>>>>>> missing
>>>>>>>> verification for the correctness of the combination of parallelism
>> and
>>>>>>>> max-parallelism. Due to lacking test coverage this introduced two
>> more
>>>>>>> bugs:
>>>>>>>> - https://issues.apache.org/jira/browse/FLINK-6188: Some
>>>>>>>> setParallelism() methods can't cope with default parallelism
>>>>>>>> - https://issues.apache.org/jira/browse/FLINK-6209:
>>>>>>>> StreamPlanEnvironment always has a parallelism of 1
>>>>>>>>
>>>>>>>> IMHO, the options are:
>>>>>>>> 1) revert the changes made for FLINK-5808 on the release-1.2 branch
>>>>>> and
>>>>>>>> live with the bug still being present
>>>>>>>> 2) put in more work to fix FLINK-5808 which requires fixing some
>>>>>>> problems
>>>>>>>> that have existed for a long time with how the parallelism is set in
>>>>>>>> streaming programs
>>>>>>>>
>>>>>>>> Best,
>>>>>>>> Aljoscha
>>>>>>>>
>>>>>>>>> On 31. Mar 2017, at 21:34, Robert Metzger <rm...@apache.org>
>>>>>> wrote:
>>>>>>>>> I don't know what is best to do, but I think releasing 1.2.1 with
>>>>>>>>> potentially more bugs than 1.2.0 is not a good option.
>>>>>>>>> I suspect a good workaround for FLINK-6188
>>>>>>>>> <https://issues.apache.org/jira/browse/FLINK-6188> is setting the
>>>>>>>>> parallelism manually for operators that can't cope with the default
>>>>>> -1
>>>>>>>>> parallelism.
>>>>>>>>>
>>>>>>>>> On Fri, Mar 31, 2017 at 9:06 PM, Aljoscha Krettek <
>>>>>> aljoscha@apache.org
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> You mean reverting the changes around FLINK-5808 [1]? This is what
>>>>>>>>>> introduced the follow-up FLINK-6188 [2].
>>>>>>>>>>
>>>>>>>>>> [1] https://issues.apache.org/jira/browse/FLINK-5808
>>>>>>>>>> [2]https://issues.apache.org/jira/browse/FLINK-6188
>>>>>>>>>>
>>>>>>>>>> On Fri, Mar 31, 2017, at 19:10, Robert Metzger wrote:
>>>>>>>>>>> I think reverting FLINK-6188 for the 1.2 branch might be a good
>>>>>> idea.
>>>>>>>>>>> FLINK-6188 introduced two new bugs, so undoing the FLINK-6188 fix
>>>>>>> will
>>>>>>>>>>> lead
>>>>>>>>>>> only to one known bug in 1.2.1, instead of an uncertain number of
>>>>>>>> issues.
>>>>>>>>>>> So 1.2.1 is not going to be worse than 1.2.0
>>>>>>>>>>>
>>>>>>>>>>> The fix will hopefully make it into 1.2.2 then.
>>>>>>>>>>>
>>>>>>>>>>> Any other thoughts on this?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Mar 31, 2017 at 6:46 PM, Fabian Hueske <
>> fhueske@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>> I merged the fix for FLINK-6044 to the release-1.2 and
>> release-1.1
>>>>>>>>>> branch.
>>>>>>>>>>>> 2017-03-31 15:02 GMT+02:00 Fabian Hueske <fh...@gmail.com>:
>>>>>>>>>>>>
>>>>>>>>>>>>> We should also backport the fix for FLINK-6044 to Flink 1.2.1.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I'll take care of that.
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2017-03-30 18:50 GMT+02:00 Aljoscha Krettek <
>> aljoscha@apache.org
>>>>>>> :
>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6188 turns out to
>>>>>> be
>>>>>>> a
>>>>>>>>>> bit
>>>>>>>>>>>>>> more involved, see my comments on the PR:
>>>>>>>>>>>>>> https://github.com/apache/flink/pull/3616.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> As I said there, maybe we should revert the commits regarding
>>>>>>>>>>>>>> parallelism/max-parallelism changes and release and then fix
>> it
>>>>>>>>>> later.
>>>>>>>>>>>>>> On Wed, Mar 29, 2017, at 23:08, Aljoscha Krettek wrote:
>>>>>>>>>>>>>>> I commented on FLINK-6214: I think it's working as intended,
>>>>>>>>>> although
>>>>>>>>>>>> we
>>>>>>>>>>>>>>> could fix the javadoc/doc.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Mar 29, 2017, at 17:35, Timo Walther wrote:
>>>>>>>>>>>>>>>> A user reported that all tumbling and slinding window
>>>>>> assigners
>>>>>>>>>>>>>> contain
>>>>>>>>>>>>>>>> a pretty obvious bug about offsets.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6214
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I think we should also fix this for 1.2.1. What do you
>> think?
>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>> Timo
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Am 29/03/17 um 11:30 schrieb Robert Metzger:
>>>>>>>>>>>>>>>>> Hi Haohui,
>>>>>>>>>>>>>>>>> I agree that we should fix the parallelism issue.
>> Otherwise,
>>>>>>>>>> the
>>>>>>>>>>>>>> 1.2.1
>>>>>>>>>>>>>>>>> release would introduce a new bug.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Tue, Mar 28, 2017 at 11:59 PM, Haohui Mai <
>>>>>>>>>> ricetons@gmail.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>> -1 (non-binding)
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> We recently found out that all jobs submitted via UI will
>>>>>>>>>> have a
>>>>>>>>>>>>>>>>>> parallelism of 1, potentially due to FLINK-5808.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Filed FLINK-6209 to track it.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> ~Haohui
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Mon, Mar 27, 2017 at 2:59 AM Chesnay Schepler <
>>>>>>>>>>>>>> chesnay@apache.org>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> If possible I would like to include FLINK-6183 &
>> FLINK-6184
>>>>>>>>>> as
>>>>>>>>>>>>>> well.
>>>>>>>>>>>>>>>>>>> They fix 2 metric-related issues that could arise when a
>>>>>>>>>> Task is
>>>>>>>>>>>>>>>>>>> cancelled very early. (like, right away)
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> FLINK-6183 fixes a memory leak where the TaskMetricGroup
>>>>>> was
>>>>>>>>>>>>>> never closed
>>>>>>>>>>>>>>>>>>> FLINK-6184 fixes a NullPointerExceptions in the buffer
>>>>>>>>>> metrics
>>>>>>>>>>>>>>>>>>> PR here: https://github.com/apache/flink/pull/3611
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On 26.03.2017 12:35, Aljoscha Krettek wrote:
>>>>>>>>>>>>>>>>>>>> I opened a PR for FLINK-6188:
>> https://github.com/apache/
>>>>>>>>>>>>>>>>>> flink/pull/3616
>>>>>>>>>>>>>>>>>>> <https://github.com/apache/flink/pull/3616>
>>>>>>>>>>>>>>>>>>>> This improves the previously very sparse test coverage
>> for
>>>>>>>>>>>>>>>>>>> timestamp/watermark assigners and fixes the bug.
>>>>>>>>>>>>>>>>>>>>> On 25 Mar 2017, at 10:22, Ufuk Celebi <uc...@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>> I agree with Aljoscha.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> -1 because of FLINK-6188
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Sat, Mar 25, 2017 at 9:38 AM, Aljoscha Krettek <
>>>>>>>>>>>>>>>>>> aljoscha@apache.org>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>> I filed this issue, which was observed by a user:
>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6188
>>>>>>>>>>>>>>>>>>>>>> I think that\u2019s blocking for 1.2.1.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On 24 Mar 2017, at 18:57, Ufuk Celebi <
>> uce@apache.org>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>> RC1 doesn't contain Stefan's backport for the
>>>>>>>>>> Asynchronous
>>>>>>>>>>>>>> snapshots
>>>>>>>>>>>>>>>>>>>>>>> for heap-based keyed state that has been merged.
>> Should
>>>>>>>>>> we
>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>>> RC2
>>>>>>>>>>>>>>>>>>>>>>> with that fix since the voting period only starts on
>>>>>>>>>> Monday?
>>>>>>>>>>>>>> I think
>>>>>>>>>>>>>>>>>>>>>>> it would only mean rerunning the scripts on your
>> side,
>>>>>>>>>>>> right?
>>>>>>>>>>>>>>>>>>>>>>> \u2013 Ufuk
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On Fri, Mar 24, 2017 at 3:05 PM, Robert Metzger <
>>>>>>>>>>>>>>>>>> rmetzger@apache.org>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>> Dear Flink community,
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Please vote on releasing the following candidate as
>>>>>>>>>> Apache
>>>>>>>>>>>>>> Flink
>>>>>>>>>>>>>>>>>>> version 1.2
>>>>>>>>>>>>>>>>>>>>>>>> .1.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> The commit to be voted on:
>>>>>>>>>>>>>>>>>>>>>>>> *732e55bd* (*
>>>>>>>>>>>>>>>>>>> http://git-wip-us.apache.org/repos/asf/flink/commit/
>>>>>>>>>> 732e55bd
>>>>>>>>>>>>>>>>>>>>>>>> <http://git-wip-us.apache.org/
>>>>>>>>>>>> repos/asf/flink/commit/732e55b
>>>>>>>>>>>>>> d>*)
>>>>>>>>>>>>>>>>>>>>>>>> Branch:
>>>>>>>>>>>>>>>>>>>>>>>> release-1.2.1-rc1
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> The release artifacts to be voted on can be found
>> at:
>>>>>>>>>>>>>>>>>>>>>>>> *http://people.apache.org/~
>> rmetzger/flink-1.2.1-rc1/
>>>>>>>>>>>>>>>>>>>>>>>> <http://people.apache.org/~
>> rmetzger/flink-1.2.1-rc1/
>>>>>>> *
>>>>>>>>>>>>>>>>>>>>>>>> The release artifacts are signed with the key with
>>>>>>>>>>>>>> fingerprint
>>>>>>>>>>>>>>>>>>> D9839159:
>>>>>>>>>>>>>>>>>>>>>>>> http://www.apache.org/dist/flink/KEYS
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> The staging repository for this release can be found
>>>>>>>>>> at:
>>>>>>>>>>>>>>>>>>> https://repository.apache.org/
>>>>>>>>>> content/repositories/orgapache
>>>>>>>>>>>>>> flink-1116
>>>>>>>>>>>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>> -
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> The vote ends on Wednesday, March 29, 2017, 3pm CET.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> [ ] +1 Release this package as Apache Flink 1.2.1
>>>>>>>>>>>>>>>>>>>>>>>> [ ] -1 Do not release this package, because ...
>>>>>>>>>>>>>
>>>>>>>>
>>>>
>>
Re: [VOTE] Release Apache Flink 1.2.1 (RC1)
Posted by Robert Metzger <rm...@apache.org>.
I've now started building the next release candidate.
On Sun, Apr 9, 2017 at 12:37 PM, Robert Metzger <rm...@apache.org> wrote:
> Hi Gyula,
>
> I'm trying to push Stefan R. to get the RocksDB fixes in asap.
>
> On Sat, Apr 8, 2017 at 5:17 PM, Gyula Fóra <gy...@gmail.com> wrote:
>
>> Hi All,
>>
>> Any updates on this?
>>
>> It would be nice to get this out soon, the Kafka bug is hurting our prod
>> jobs big time.
>>
>> Thanks,
>> Gyula
>>
>> On Wed, Apr 5, 2017, 15:27 Ufuk Celebi <uc...@apache.org> wrote:
>>
>> > @Stefan: What's the state with the RocksDB fixes? I would be +1 to do
>> this.
>> >
>> > On Tue, Apr 4, 2017 at 6:05 PM, Chesnay Schepler <ch...@apache.org>
>> > wrote:
>> > > Yes, aljoscha already opened one against master:
>> > > https://github.com/apache/flink/pull/3670
>> > >
>> > > On 04.04.2017 17:57, Ted Yu wrote:
>> > >>
>> > >> Should the commits be reverted from master branch as well ?
>> > >>
>> > >> On Tue, Apr 4, 2017 at 4:59 AM, Aljoscha Krettek <
>> aljoscha@apache.org>
>> > >> wrote:
>> > >>
>> > >>> The commits around FLINK-5808 have been reverted on release-1.2.
>> > >>>
>> > >>>> On 4. Apr 2017, at 12:16, Stefan Richter <
>> s.richter@data-artisans.com
>> > >
>> > >>>
>> > >>> wrote:
>> > >>>>
>> > >>>> I have created a custom build of RocksDB 4.11.2 that fixes a
>> > significant
>> > >>>
>> > >>> performance problem with append operations. I think this should
>> > >>> definitely
>> > >>> be part of the 1.2.1 release because this is already blocking some
>> > users.
>> > >>> What is missing is uploading the jar to maven central and a testing
>> > run,
>> > >>> e.g. with some misbehaved job that has large state.
>> > >>>>
>> > >>>>
>> > >>>>> Am 04.04.2017 um 11:57 schrieb Robert Metzger <
>> rmetzger@apache.org>:
>> > >>>>>
>> > >>>>> Thank you for opening a PR for this.
>> > >>>>>
>> > >>>>> Chesnay, do you need more reviews for the metrics changes /
>> > backports?
>> > >>>>>
>> > >>>>> Are there any other release blockers for 1.2.1, or are we good to
>> go?
>> > >>>>>
>> > >>>>> On Mon, Apr 3, 2017 at 6:48 PM, Aljoscha Krettek <
>> > aljoscha@apache.org>
>> > >>>>> wrote:
>> > >>>>>
>> > >>>>>> I created a PR for the revert: https://github.com/apache/
>> > >>>
>> > >>> flink/pull/3664
>> > >>>>>>>
>> > >>>>>>> On 3. Apr 2017, at 18:32, Stephan Ewen <se...@apache.org>
>> wrote:
>> > >>>>>>>
>> > >>>>>>> +1 for options (1), but also invest the time to fix it properly
>> for
>> > >>>
>> > >>> 1.2.2
>> > >>>>>>>
>> > >>>>>>>
>> > >>>>>>> On Mon, Apr 3, 2017 at 9:10 AM, Kostas Kloudas <
>> > >>>>>>
>> > >>>>>> k.kloudas@data-artisans.com>
>> > >>>>>>>
>> > >>>>>>> wrote:
>> > >>>>>>>
>> > >>>>>>>> +1 for 1
>> > >>>>>>>>
>> > >>>>>>>>> On Apr 3, 2017, at 5:52 PM, Till Rohrmann <
>> trohrmann@apache.org>
>> > >>>>>>
>> > >>>>>> wrote:
>> > >>>>>>>>>
>> > >>>>>>>>> +1 for option 1)
>> > >>>>>>>>>
>> > >>>>>>>>> On Mon, Apr 3, 2017 at 5:48 PM, Fabian Hueske <
>> fhueske@gmail.com
>> > >
>> > >>>>>>
>> > >>>>>> wrote:
>> > >>>>>>>>>>
>> > >>>>>>>>>> +1 to option 1)
>> > >>>>>>>>>>
>> > >>>>>>>>>> 2017-04-03 16:57 GMT+02:00 Ted Yu <yu...@gmail.com>:
>> > >>>>>>>>>>
>> > >>>>>>>>>>> Looks like #1 is better - 1.2.1 would be at least as stable
>> as
>> > >>>
>> > >>> 1.2.0
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> Cheers
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> On Mon, Apr 3, 2017 at 7:39 AM, Aljoscha Krettek <
>> > >>>>>>
>> > >>>>>> aljoscha@apache.org>
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> wrote:
>> > >>>>>>>>>>>
>> > >>>>>>>>>>>> Just so we’re all on the same page. ;-)
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> There was https://issues.apache.org/jira/browse/FLINK-5808
>> > which
>> > >>>>>>
>> > >>>>>> was
>> > >>>>>>>>
>> > >>>>>>>> a
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> bug that we initially discovered in Flink 1.2 which was/is
>> > about
>> > >>>>>>>>>>
>> > >>>>>>>>>> missing
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> verification for the correctness of the combination of
>> > >>>
>> > >>> parallelism
>> > >>>>>>
>> > >>>>>> and
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> max-parallelism. Due to lacking test coverage this
>> introduced
>> > >>>>>>>>>>>> two
>> > >>>>>>
>> > >>>>>> more
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> bugs:
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> - https://issues.apache.org/jira/browse/FLINK-6188: Some
>> > >>>>>>>>>>>> setParallelism() methods can't cope with default
>> parallelism
>> > >>>>>>>>>>>> - https://issues.apache.org/jira/browse/FLINK-6209:
>> > >>>>>>>>>>>> StreamPlanEnvironment always has a parallelism of 1
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> IMHO, the options are:
>> > >>>>>>>>>>>> 1) revert the changes made for FLINK-5808 on the
>> release-1.2
>> > >>>
>> > >>> branch
>> > >>>>>>>>>>
>> > >>>>>>>>>> and
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> live with the bug still being present
>> > >>>>>>>>>>>> 2) put in more work to fix FLINK-5808 which requires fixing
>> > some
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> problems
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> that have existed for a long time with how the parallelism
>> is
>> > >>>
>> > >>> set in
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> streaming programs
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> Best,
>> > >>>>>>>>>>>> Aljoscha
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>>> On 31. Mar 2017, at 21:34, Robert Metzger <
>> > rmetzger@apache.org>
>> > >>>>>>>>>>
>> > >>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> I don't know what is best to do, but I think releasing
>> 1.2.1
>> > >>>
>> > >>> with
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> potentially more bugs than 1.2.0 is not a good option.
>> > >>>>>>>>>>>>> I suspect a good workaround for FLINK-6188
>> > >>>>>>>>>>>>> <https://issues.apache.org/jira/browse/FLINK-6188> is
>> > setting
>> > >>>
>> > >>> the
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> parallelism manually for operators that can't cope with
>> the
>> > >>>
>> > >>> default
>> > >>>>>>>>>>
>> > >>>>>>>>>> -1
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> parallelism.
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> On Fri, Mar 31, 2017 at 9:06 PM, Aljoscha Krettek <
>> > >>>>>>>>>>
>> > >>>>>>>>>> aljoscha@apache.org
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> You mean reverting the changes around FLINK-5808 [1]?
>> This
>> > is
>> > >>>
>> > >>> what
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> introduced the follow-up FLINK-6188 [2].
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/FLINK-5808
>> > >>>>>>>>>>>>>> [2]https://issues.apache.org/jira/browse/FLINK-6188
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> On Fri, Mar 31, 2017, at 19:10, Robert Metzger wrote:
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>> I think reverting FLINK-6188 for the 1.2 branch might
>> be a
>> > >>>
>> > >>> good
>> > >>>>>>>>>>
>> > >>>>>>>>>> idea.
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>> FLINK-6188 introduced two new bugs, so undoing the
>> > FLINK-6188
>> > >>>
>> > >>> fix
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> will
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>> lead
>> > >>>>>>>>>>>>>>> only to one known bug in 1.2.1, instead of an uncertain
>> > >>>
>> > >>> number of
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> issues.
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>> So 1.2.1 is not going to be worse than 1.2.0
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>> The fix will hopefully make it into 1.2.2 then.
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>> Any other thoughts on this?
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>> On Fri, Mar 31, 2017 at 6:46 PM, Fabian Hueske <
>> > >>>>>>
>> > >>>>>> fhueske@gmail.com>
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>> I merged the fix for FLINK-6044 to the release-1.2 and
>> > >>>>>>
>> > >>>>>> release-1.1
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> branch.
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>> 2017-03-31 15:02 GMT+02:00 Fabian Hueske <
>> > fhueske@gmail.com
>> > >>>>
>> > >>>> :
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>> We should also backport the fix for FLINK-6044 to
>> Flink
>> > >>>
>> > >>> 1.2.1.
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>> I'll take care of that.
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>> 2017-03-30 18:50 GMT+02:00 Aljoscha Krettek <
>> > >>>>>>
>> > >>>>>> aljoscha@apache.org
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> :
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6188
>> turns
>> > >>>
>> > >>> out to
>> > >>>>>>>>>>
>> > >>>>>>>>>> be
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> a
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> bit
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> more involved, see my comments on the PR:
>> > >>>>>>>>>>>>>>>>>> https://github.com/apache/flink/pull/3616.
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> As I said there, maybe we should revert the commits
>> > >>>
>> > >>> regarding
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> parallelism/max-parallelism changes and release and
>> then
>> > >>>
>> > >>> fix
>> > >>>>>>
>> > >>>>>> it
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> later.
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2017, at 23:08, Aljoscha Krettek
>> wrote:
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>> I commented on FLINK-6214: I think it's working as
>> > >>>
>> > >>> intended,
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> although
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>> we
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>> could fix the javadoc/doc.
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2017, at 17:35, Timo Walther wrote:
>> > >>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>> A user reported that all tumbling and slinding
>> window
>> > >>>>>>>>>>
>> > >>>>>>>>>> assigners
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> contain
>> > >>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>> a pretty obvious bug about offsets.
>> > >>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6214
>> > >>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>> I think we should also fix this for 1.2.1. What do
>> you
>> > >>>>>>
>> > >>>>>> think?
>> > >>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>> Regards,
>> > >>>>>>>>>>>>>>>>>>>> Timo
>> > >>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>> Am 29/03/17 um 11:30 schrieb Robert Metzger:
>> > >>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>> Hi Haohui,
>> > >>>>>>>>>>>>>>>>>>>>> I agree that we should fix the parallelism issue.
>> > >>>>>>
>> > >>>>>> Otherwise,
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> the
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> 1.2.1
>> > >>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>> release would introduce a new bug.
>> > >>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>> On Tue, Mar 28, 2017 at 11:59 PM, Haohui Mai <
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> ricetons@gmail.com>
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>> -1 (non-binding)
>> > >>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>> We recently found out that all jobs submitted
>> via UI
>> > >>>
>> > >>> will
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> have a
>> > >>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>> parallelism of 1, potentially due to FLINK-5808.
>> > >>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>> Filed FLINK-6209 to track it.
>> > >>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>> ~Haohui
>> > >>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>> On Mon, Mar 27, 2017 at 2:59 AM Chesnay Schepler
>> <
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> chesnay@apache.org>
>> > >>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>> If possible I would like to include FLINK-6183 &
>> > >>>>>>
>> > >>>>>> FLINK-6184
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> as
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> well.
>> > >>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>> They fix 2 metric-related issues that could
>> arise
>> > >>>
>> > >>> when a
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> Task is
>> > >>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>> cancelled very early. (like, right away)
>> > >>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>> FLINK-6183 fixes a memory leak where the
>> > >>>
>> > >>> TaskMetricGroup
>> > >>>>>>>>>>
>> > >>>>>>>>>> was
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> never closed
>> > >>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>> FLINK-6184 fixes a NullPointerExceptions in the
>> > >>>>>>>>>>>>>>>>>>>>>>> buffer
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> metrics
>> > >>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>> PR here: https://github.com/apache/flin
>> k/pull/3611
>> > >>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>> On 26.03.2017 12:35, Aljoscha Krettek wrote:
>> > >>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>> I opened a PR for FLINK-6188:
>> > >>>>>>
>> > >>>>>> https://github.com/apache/
>> > >>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>> flink/pull/3616
>> > >>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>> <https://github.com/apache/flink/pull/3616>
>> > >>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>> This improves the previously very sparse test
>> > >>>
>> > >>> coverage
>> > >>>>>>
>> > >>>>>> for
>> > >>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>> timestamp/watermark assigners and fixes the bug.
>> > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>> On 25 Mar 2017, at 10:22, Ufuk Celebi <
>> > >>>
>> > >>> uce@apache.org>
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>> I agree with Aljoscha.
>> > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>> -1 because of FLINK-6188
>> > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>> On Sat, Mar 25, 2017 at 9:38 AM, Aljoscha
>> > Krettek <
>> > >>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>> aljoscha@apache.org>
>> > >>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>> I filed this issue, which was observed by a
>> > user:
>> > >>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira
>> /browse/FLINK-6188
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>> I think that’s blocking for 1.2.1.
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> On 24 Mar 2017, at 18:57, Ufuk Celebi <
>> > >>>>>>
>> > >>>>>> uce@apache.org>
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> RC1 doesn't contain Stefan's backport for
>> the
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> Asynchronous
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> snapshots
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> for heap-based keyed state that has been
>> > merged.
>> > >>>>>>
>> > >>>>>> Should
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> we
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> create
>> > >>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>> RC2
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> with that fix since the voting period only
>> > starts
>> > >>>
>> > >>> on
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> Monday?
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> I think
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> it would only mean rerunning the scripts on
>> > your
>> > >>>>>>
>> > >>>>>> side,
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>> right?
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> – Ufuk
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Mar 24, 2017 at 3:05 PM, Robert
>> > Metzger <
>> > >>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>> rmetzger@apache.org>
>> > >>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Dear Flink community,
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Please vote on releasing the following
>> > candidate
>> > >>>
>> > >>> as
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> Apache
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> Flink
>> > >>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>> version 1.2
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> .1.
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> The commit to be voted on:
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> *732e55bd* (*
>> > >>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>
>> > http://git-wip-us.apache.org/repos/asf/flink/commit/
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> 732e55bd
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> <http://git-wip-us.apache.org/
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>> repos/asf/flink/commit/732e55b
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> d>*)
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Branch:
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> release-1.2.1-rc1
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> The release artifacts to be voted on can be
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> found
>> > >>>>>>
>> > >>>>>> at:
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> *http://people.apache.org/~
>> > >>>>>>
>> > >>>>>> rmetzger/flink-1.2.1-rc1/
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> <http://people.apache.org/~
>> > >>>>>>
>> > >>>>>> rmetzger/flink-1.2.1-rc1/
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> *
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> The release artifacts are signed with the
>> key
>> > >>>
>> > >>> with
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> fingerprint
>> > >>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>> D9839159:
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> http://www.apache.org/dist/flink/KEYS
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> The staging repository for this release
>> can be
>> > >>>
>> > >>> found
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> at:
>> > >>>>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>>>> https://repository.apache.org/
>> >
>>
>
>
Re: [VOTE] Release Apache Flink 1.2.1 (RC1)
Posted by Robert Metzger <rm...@apache.org>.
Hi Gyula,
I'm trying to push Stefan R. to get the RocksDB fixes in asap.
On Sat, Apr 8, 2017 at 5:17 PM, Gyula Fóra <gy...@gmail.com> wrote:
> Hi All,
>
> Any updates on this?
>
> It would be nice to get this out soon, the Kafka bug is hurting our prod
> jobs big time.
>
> Thanks,
> Gyula
>
> On Wed, Apr 5, 2017, 15:27 Ufuk Celebi <uc...@apache.org> wrote:
>
> > @Stefan: What's the state with the RocksDB fixes? I would be +1 to do
> this.
> >
> > On Tue, Apr 4, 2017 at 6:05 PM, Chesnay Schepler <ch...@apache.org>
> > wrote:
> > > Yes, aljoscha already opened one against master:
> > > https://github.com/apache/flink/pull/3670
> > >
> > > On 04.04.2017 17:57, Ted Yu wrote:
> > >>
> > >> Should the commits be reverted from master branch as well ?
> > >>
> > >> On Tue, Apr 4, 2017 at 4:59 AM, Aljoscha Krettek <aljoscha@apache.org
> >
> > >> wrote:
> > >>
> > >>> The commits around FLINK-5808 have been reverted on release-1.2.
> > >>>
> > >>>> On 4. Apr 2017, at 12:16, Stefan Richter <
> s.richter@data-artisans.com
> > >
> > >>>
> > >>> wrote:
> > >>>>
> > >>>> I have created a custom build of RocksDB 4.11.2 that fixes a
> > significant
> > >>>
> > >>> performance problem with append operations. I think this should
> > >>> definitely
> > >>> be part of the 1.2.1 release because this is already blocking some
> > users.
> > >>> What is missing is uploading the jar to maven central and a testing
> > run,
> > >>> e.g. with some misbehaved job that has large state.
> > >>>>
> > >>>>
> > >>>>> Am 04.04.2017 um 11:57 schrieb Robert Metzger <rmetzger@apache.org
> >:
> > >>>>>
> > >>>>> Thank you for opening a PR for this.
> > >>>>>
> > >>>>> Chesnay, do you need more reviews for the metrics changes /
> > backports?
> > >>>>>
> > >>>>> Are there any other release blockers for 1.2.1, or are we good to
> go?
> > >>>>>
> > >>>>> On Mon, Apr 3, 2017 at 6:48 PM, Aljoscha Krettek <
> > aljoscha@apache.org>
> > >>>>> wrote:
> > >>>>>
> > >>>>>> I created a PR for the revert: https://github.com/apache/
> > >>>
> > >>> flink/pull/3664
> > >>>>>>>
> > >>>>>>> On 3. Apr 2017, at 18:32, Stephan Ewen <se...@apache.org> wrote:
> > >>>>>>>
> > >>>>>>> +1 for options (1), but also invest the time to fix it properly
> for
> > >>>
> > >>> 1.2.2
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> On Mon, Apr 3, 2017 at 9:10 AM, Kostas Kloudas <
> > >>>>>>
> > >>>>>> k.kloudas@data-artisans.com>
> > >>>>>>>
> > >>>>>>> wrote:
> > >>>>>>>
> > >>>>>>>> +1 for 1
> > >>>>>>>>
> > >>>>>>>>> On Apr 3, 2017, at 5:52 PM, Till Rohrmann <
> trohrmann@apache.org>
> > >>>>>>
> > >>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>> +1 for option 1)
> > >>>>>>>>>
> > >>>>>>>>> On Mon, Apr 3, 2017 at 5:48 PM, Fabian Hueske <
> fhueske@gmail.com
> > >
> > >>>>>>
> > >>>>>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>> +1 to option 1)
> > >>>>>>>>>>
> > >>>>>>>>>> 2017-04-03 16:57 GMT+02:00 Ted Yu <yu...@gmail.com>:
> > >>>>>>>>>>
> > >>>>>>>>>>> Looks like #1 is better - 1.2.1 would be at least as stable
> as
> > >>>
> > >>> 1.2.0
> > >>>>>>>>>>>
> > >>>>>>>>>>> Cheers
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Mon, Apr 3, 2017 at 7:39 AM, Aljoscha Krettek <
> > >>>>>>
> > >>>>>> aljoscha@apache.org>
> > >>>>>>>>>>>
> > >>>>>>>>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>>> Just so we’re all on the same page. ;-)
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> There was https://issues.apache.org/jira/browse/FLINK-5808
> > which
> > >>>>>>
> > >>>>>> was
> > >>>>>>>>
> > >>>>>>>> a
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> bug that we initially discovered in Flink 1.2 which was/is
> > about
> > >>>>>>>>>>
> > >>>>>>>>>> missing
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> verification for the correctness of the combination of
> > >>>
> > >>> parallelism
> > >>>>>>
> > >>>>>> and
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> max-parallelism. Due to lacking test coverage this
> introduced
> > >>>>>>>>>>>> two
> > >>>>>>
> > >>>>>> more
> > >>>>>>>>>>>
> > >>>>>>>>>>> bugs:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> - https://issues.apache.org/jira/browse/FLINK-6188: Some
> > >>>>>>>>>>>> setParallelism() methods can't cope with default parallelism
> > >>>>>>>>>>>> - https://issues.apache.org/jira/browse/FLINK-6209:
> > >>>>>>>>>>>> StreamPlanEnvironment always has a parallelism of 1
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> IMHO, the options are:
> > >>>>>>>>>>>> 1) revert the changes made for FLINK-5808 on the release-1.2
> > >>>
> > >>> branch
> > >>>>>>>>>>
> > >>>>>>>>>> and
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> live with the bug still being present
> > >>>>>>>>>>>> 2) put in more work to fix FLINK-5808 which requires fixing
> > some
> > >>>>>>>>>>>
> > >>>>>>>>>>> problems
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> that have existed for a long time with how the parallelism
> is
> > >>>
> > >>> set in
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> streaming programs
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Best,
> > >>>>>>>>>>>> Aljoscha
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> On 31. Mar 2017, at 21:34, Robert Metzger <
> > rmetzger@apache.org>
> > >>>>>>>>>>
> > >>>>>>>>>> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> I don't know what is best to do, but I think releasing
> 1.2.1
> > >>>
> > >>> with
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> potentially more bugs than 1.2.0 is not a good option.
> > >>>>>>>>>>>>> I suspect a good workaround for FLINK-6188
> > >>>>>>>>>>>>> <https://issues.apache.org/jira/browse/FLINK-6188> is
> > setting
> > >>>
> > >>> the
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> parallelism manually for operators that can't cope with the
> > >>>
> > >>> default
> > >>>>>>>>>>
> > >>>>>>>>>> -1
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> parallelism.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> On Fri, Mar 31, 2017 at 9:06 PM, Aljoscha Krettek <
> > >>>>>>>>>>
> > >>>>>>>>>> aljoscha@apache.org
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>> You mean reverting the changes around FLINK-5808 [1]? This
> > is
> > >>>
> > >>> what
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> introduced the follow-up FLINK-6188 [2].
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/FLINK-5808
> > >>>>>>>>>>>>>> [2]https://issues.apache.org/jira/browse/FLINK-6188
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> On Fri, Mar 31, 2017, at 19:10, Robert Metzger wrote:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> I think reverting FLINK-6188 for the 1.2 branch might be
> a
> > >>>
> > >>> good
> > >>>>>>>>>>
> > >>>>>>>>>> idea.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> FLINK-6188 introduced two new bugs, so undoing the
> > FLINK-6188
> > >>>
> > >>> fix
> > >>>>>>>>>>>
> > >>>>>>>>>>> will
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> lead
> > >>>>>>>>>>>>>>> only to one known bug in 1.2.1, instead of an uncertain
> > >>>
> > >>> number of
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> issues.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> So 1.2.1 is not going to be worse than 1.2.0
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> The fix will hopefully make it into 1.2.2 then.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Any other thoughts on this?
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> On Fri, Mar 31, 2017 at 6:46 PM, Fabian Hueske <
> > >>>>>>
> > >>>>>> fhueske@gmail.com>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> I merged the fix for FLINK-6044 to the release-1.2 and
> > >>>>>>
> > >>>>>> release-1.1
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> branch.
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> 2017-03-31 15:02 GMT+02:00 Fabian Hueske <
> > fhueske@gmail.com
> > >>>>
> > >>>> :
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> We should also backport the fix for FLINK-6044 to Flink
> > >>>
> > >>> 1.2.1.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> I'll take care of that.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> 2017-03-30 18:50 GMT+02:00 Aljoscha Krettek <
> > >>>>>>
> > >>>>>> aljoscha@apache.org
> > >>>>>>>>>>>
> > >>>>>>>>>>> :
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6188
> turns
> > >>>
> > >>> out to
> > >>>>>>>>>>
> > >>>>>>>>>> be
> > >>>>>>>>>>>
> > >>>>>>>>>>> a
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> bit
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> more involved, see my comments on the PR:
> > >>>>>>>>>>>>>>>>>> https://github.com/apache/flink/pull/3616.
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> As I said there, maybe we should revert the commits
> > >>>
> > >>> regarding
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> parallelism/max-parallelism changes and release and
> then
> > >>>
> > >>> fix
> > >>>>>>
> > >>>>>> it
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> later.
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2017, at 23:08, Aljoscha Krettek
> wrote:
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> I commented on FLINK-6214: I think it's working as
> > >>>
> > >>> intended,
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> although
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> we
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> could fix the javadoc/doc.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2017, at 17:35, Timo Walther wrote:
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> A user reported that all tumbling and slinding
> window
> > >>>>>>>>>>
> > >>>>>>>>>> assigners
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> contain
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> a pretty obvious bug about offsets.
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6214
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> I think we should also fix this for 1.2.1. What do
> you
> > >>>>>>
> > >>>>>> think?
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> Regards,
> > >>>>>>>>>>>>>>>>>>>> Timo
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> Am 29/03/17 um 11:30 schrieb Robert Metzger:
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> Hi Haohui,
> > >>>>>>>>>>>>>>>>>>>>> I agree that we should fix the parallelism issue.
> > >>>>>>
> > >>>>>> Otherwise,
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> 1.2.1
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> release would introduce a new bug.
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> On Tue, Mar 28, 2017 at 11:59 PM, Haohui Mai <
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> ricetons@gmail.com>
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> -1 (non-binding)
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> We recently found out that all jobs submitted via
> UI
> > >>>
> > >>> will
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> have a
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> parallelism of 1, potentially due to FLINK-5808.
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> Filed FLINK-6209 to track it.
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> ~Haohui
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> On Mon, Mar 27, 2017 at 2:59 AM Chesnay Schepler <
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> chesnay@apache.org>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> If possible I would like to include FLINK-6183 &
> > >>>>>>
> > >>>>>> FLINK-6184
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> as
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> well.
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> They fix 2 metric-related issues that could arise
> > >>>
> > >>> when a
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Task is
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> cancelled very early. (like, right away)
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> FLINK-6183 fixes a memory leak where the
> > >>>
> > >>> TaskMetricGroup
> > >>>>>>>>>>
> > >>>>>>>>>> was
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> never closed
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> FLINK-6184 fixes a NullPointerExceptions in the
> > >>>>>>>>>>>>>>>>>>>>>>> buffer
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> metrics
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> PR here: https://github.com/apache/
> flink/pull/3611
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> On 26.03.2017 12:35, Aljoscha Krettek wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> I opened a PR for FLINK-6188:
> > >>>>>>
> > >>>>>> https://github.com/apache/
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> flink/pull/3616
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> <https://github.com/apache/flink/pull/3616>
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> This improves the previously very sparse test
> > >>>
> > >>> coverage
> > >>>>>>
> > >>>>>> for
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> timestamp/watermark assigners and fixes the bug.
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>> On 25 Mar 2017, at 10:22, Ufuk Celebi <
> > >>>
> > >>> uce@apache.org>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>> I agree with Aljoscha.
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>> -1 because of FLINK-6188
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>> On Sat, Mar 25, 2017 at 9:38 AM, Aljoscha
> > Krettek <
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> aljoscha@apache.org>
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> I filed this issue, which was observed by a
> > user:
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6188
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>> I think that’s blocking for 1.2.1.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> On 24 Mar 2017, at 18:57, Ufuk Celebi <
> > >>>>>>
> > >>>>>> uce@apache.org>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> RC1 doesn't contain Stefan's backport for the
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Asynchronous
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> snapshots
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> for heap-based keyed state that has been
> > merged.
> > >>>>>>
> > >>>>>> Should
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> we
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> create
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> RC2
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> with that fix since the voting period only
> > starts
> > >>>
> > >>> on
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Monday?
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> I think
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> it would only mean rerunning the scripts on
> > your
> > >>>>>>
> > >>>>>> side,
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> right?
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> – Ufuk
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Mar 24, 2017 at 3:05 PM, Robert
> > Metzger <
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> rmetzger@apache.org>
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Dear Flink community,
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Please vote on releasing the following
> > candidate
> > >>>
> > >>> as
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Apache
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> Flink
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> version 1.2
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> .1.
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> The commit to be voted on:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> *732e55bd* (*
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>
> > http://git-wip-us.apache.org/repos/asf/flink/commit/
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> 732e55bd
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> <http://git-wip-us.apache.org/
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> repos/asf/flink/commit/732e55b
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> d>*)
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> Branch:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> release-1.2.1-rc1
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> The release artifacts to be voted on can be
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> found
> > >>>>>>
> > >>>>>> at:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> *http://people.apache.org/~
> > >>>>>>
> > >>>>>> rmetzger/flink-1.2.1-rc1/
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> <http://people.apache.org/~
> > >>>>>>
> > >>>>>> rmetzger/flink-1.2.1-rc1/
> > >>>>>>>>>>>
> > >>>>>>>>>>> *
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> The release artifacts are signed with the
> key
> > >>>
> > >>> with
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> fingerprint
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> D9839159:
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> http://www.apache.org/dist/flink/KEYS
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> The staging repository for this release can
> be
> > >>>
> > >>> found
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> at:
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> https://repository.apache.org/
> >
>
Re: [VOTE] Release Apache Flink 1.2.1 (RC1)
Posted by Gyula Fóra <gy...@gmail.com>.
Hi All,
Any updates on this?
It would be nice to get this out soon, the Kafka bug is hurting our prod
jobs big time.
Thanks,
Gyula
On Wed, Apr 5, 2017, 15:27 Ufuk Celebi <uc...@apache.org> wrote:
> @Stefan: What's the state with the RocksDB fixes? I would be +1 to do this.
>
> On Tue, Apr 4, 2017 at 6:05 PM, Chesnay Schepler <ch...@apache.org>
> wrote:
> > Yes, aljoscha already opened one against master:
> > https://github.com/apache/flink/pull/3670
> >
> > On 04.04.2017 17:57, Ted Yu wrote:
> >>
> >> Should the commits be reverted from master branch as well ?
> >>
> >> On Tue, Apr 4, 2017 at 4:59 AM, Aljoscha Krettek <al...@apache.org>
> >> wrote:
> >>
> >>> The commits around FLINK-5808 have been reverted on release-1.2.
> >>>
> >>>> On 4. Apr 2017, at 12:16, Stefan Richter <s.richter@data-artisans.com
> >
> >>>
> >>> wrote:
> >>>>
> >>>> I have created a custom build of RocksDB 4.11.2 that fixes a
> significant
> >>>
> >>> performance problem with append operations. I think this should
> >>> definitely
> >>> be part of the 1.2.1 release because this is already blocking some
> users.
> >>> What is missing is uploading the jar to maven central and a testing
> run,
> >>> e.g. with some misbehaved job that has large state.
> >>>>
> >>>>
> >>>>> Am 04.04.2017 um 11:57 schrieb Robert Metzger <rm...@apache.org>:
> >>>>>
> >>>>> Thank you for opening a PR for this.
> >>>>>
> >>>>> Chesnay, do you need more reviews for the metrics changes /
> backports?
> >>>>>
> >>>>> Are there any other release blockers for 1.2.1, or are we good to go?
> >>>>>
> >>>>> On Mon, Apr 3, 2017 at 6:48 PM, Aljoscha Krettek <
> aljoscha@apache.org>
> >>>>> wrote:
> >>>>>
> >>>>>> I created a PR for the revert: https://github.com/apache/
> >>>
> >>> flink/pull/3664
> >>>>>>>
> >>>>>>> On 3. Apr 2017, at 18:32, Stephan Ewen <se...@apache.org> wrote:
> >>>>>>>
> >>>>>>> +1 for options (1), but also invest the time to fix it properly for
> >>>
> >>> 1.2.2
> >>>>>>>
> >>>>>>>
> >>>>>>> On Mon, Apr 3, 2017 at 9:10 AM, Kostas Kloudas <
> >>>>>>
> >>>>>> k.kloudas@data-artisans.com>
> >>>>>>>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> +1 for 1
> >>>>>>>>
> >>>>>>>>> On Apr 3, 2017, at 5:52 PM, Till Rohrmann <tr...@apache.org>
> >>>>>>
> >>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>> +1 for option 1)
> >>>>>>>>>
> >>>>>>>>> On Mon, Apr 3, 2017 at 5:48 PM, Fabian Hueske <fhueske@gmail.com
> >
> >>>>>>
> >>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> +1 to option 1)
> >>>>>>>>>>
> >>>>>>>>>> 2017-04-03 16:57 GMT+02:00 Ted Yu <yu...@gmail.com>:
> >>>>>>>>>>
> >>>>>>>>>>> Looks like #1 is better - 1.2.1 would be at least as stable as
> >>>
> >>> 1.2.0
> >>>>>>>>>>>
> >>>>>>>>>>> Cheers
> >>>>>>>>>>>
> >>>>>>>>>>> On Mon, Apr 3, 2017 at 7:39 AM, Aljoscha Krettek <
> >>>>>>
> >>>>>> aljoscha@apache.org>
> >>>>>>>>>>>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Just so we’re all on the same page. ;-)
> >>>>>>>>>>>>
> >>>>>>>>>>>> There was https://issues.apache.org/jira/browse/FLINK-5808
> which
> >>>>>>
> >>>>>> was
> >>>>>>>>
> >>>>>>>> a
> >>>>>>>>>>>>
> >>>>>>>>>>>> bug that we initially discovered in Flink 1.2 which was/is
> about
> >>>>>>>>>>
> >>>>>>>>>> missing
> >>>>>>>>>>>>
> >>>>>>>>>>>> verification for the correctness of the combination of
> >>>
> >>> parallelism
> >>>>>>
> >>>>>> and
> >>>>>>>>>>>>
> >>>>>>>>>>>> max-parallelism. Due to lacking test coverage this introduced
> >>>>>>>>>>>> two
> >>>>>>
> >>>>>> more
> >>>>>>>>>>>
> >>>>>>>>>>> bugs:
> >>>>>>>>>>>>
> >>>>>>>>>>>> - https://issues.apache.org/jira/browse/FLINK-6188: Some
> >>>>>>>>>>>> setParallelism() methods can't cope with default parallelism
> >>>>>>>>>>>> - https://issues.apache.org/jira/browse/FLINK-6209:
> >>>>>>>>>>>> StreamPlanEnvironment always has a parallelism of 1
> >>>>>>>>>>>>
> >>>>>>>>>>>> IMHO, the options are:
> >>>>>>>>>>>> 1) revert the changes made for FLINK-5808 on the release-1.2
> >>>
> >>> branch
> >>>>>>>>>>
> >>>>>>>>>> and
> >>>>>>>>>>>>
> >>>>>>>>>>>> live with the bug still being present
> >>>>>>>>>>>> 2) put in more work to fix FLINK-5808 which requires fixing
> some
> >>>>>>>>>>>
> >>>>>>>>>>> problems
> >>>>>>>>>>>>
> >>>>>>>>>>>> that have existed for a long time with how the parallelism is
> >>>
> >>> set in
> >>>>>>>>>>>>
> >>>>>>>>>>>> streaming programs
> >>>>>>>>>>>>
> >>>>>>>>>>>> Best,
> >>>>>>>>>>>> Aljoscha
> >>>>>>>>>>>>
> >>>>>>>>>>>>> On 31. Mar 2017, at 21:34, Robert Metzger <
> rmetzger@apache.org>
> >>>>>>>>>>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I don't know what is best to do, but I think releasing 1.2.1
> >>>
> >>> with
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> potentially more bugs than 1.2.0 is not a good option.
> >>>>>>>>>>>>> I suspect a good workaround for FLINK-6188
> >>>>>>>>>>>>> <https://issues.apache.org/jira/browse/FLINK-6188> is
> setting
> >>>
> >>> the
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> parallelism manually for operators that can't cope with the
> >>>
> >>> default
> >>>>>>>>>>
> >>>>>>>>>> -1
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> parallelism.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Fri, Mar 31, 2017 at 9:06 PM, Aljoscha Krettek <
> >>>>>>>>>>
> >>>>>>>>>> aljoscha@apache.org
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> You mean reverting the changes around FLINK-5808 [1]? This
> is
> >>>
> >>> what
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> introduced the follow-up FLINK-6188 [2].
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/FLINK-5808
> >>>>>>>>>>>>>> [2]https://issues.apache.org/jira/browse/FLINK-6188
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Fri, Mar 31, 2017, at 19:10, Robert Metzger wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I think reverting FLINK-6188 for the 1.2 branch might be a
> >>>
> >>> good
> >>>>>>>>>>
> >>>>>>>>>> idea.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> FLINK-6188 introduced two new bugs, so undoing the
> FLINK-6188
> >>>
> >>> fix
> >>>>>>>>>>>
> >>>>>>>>>>> will
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> lead
> >>>>>>>>>>>>>>> only to one known bug in 1.2.1, instead of an uncertain
> >>>
> >>> number of
> >>>>>>>>>>>>
> >>>>>>>>>>>> issues.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> So 1.2.1 is not going to be worse than 1.2.0
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> The fix will hopefully make it into 1.2.2 then.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Any other thoughts on this?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Fri, Mar 31, 2017 at 6:46 PM, Fabian Hueske <
> >>>>>>
> >>>>>> fhueske@gmail.com>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I merged the fix for FLINK-6044 to the release-1.2 and
> >>>>>>
> >>>>>> release-1.1
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> branch.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> 2017-03-31 15:02 GMT+02:00 Fabian Hueske <
> fhueske@gmail.com
> >>>>
> >>>> :
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> We should also backport the fix for FLINK-6044 to Flink
> >>>
> >>> 1.2.1.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I'll take care of that.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> 2017-03-30 18:50 GMT+02:00 Aljoscha Krettek <
> >>>>>>
> >>>>>> aljoscha@apache.org
> >>>>>>>>>>>
> >>>>>>>>>>> :
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6188 turns
> >>>
> >>> out to
> >>>>>>>>>>
> >>>>>>>>>> be
> >>>>>>>>>>>
> >>>>>>>>>>> a
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> bit
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> more involved, see my comments on the PR:
> >>>>>>>>>>>>>>>>>> https://github.com/apache/flink/pull/3616.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> As I said there, maybe we should revert the commits
> >>>
> >>> regarding
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> parallelism/max-parallelism changes and release and then
> >>>
> >>> fix
> >>>>>>
> >>>>>> it
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> later.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2017, at 23:08, Aljoscha Krettek wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> I commented on FLINK-6214: I think it's working as
> >>>
> >>> intended,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> although
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> could fix the javadoc/doc.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2017, at 17:35, Timo Walther wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> A user reported that all tumbling and slinding window
> >>>>>>>>>>
> >>>>>>>>>> assigners
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> contain
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> a pretty obvious bug about offsets.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6214
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> I think we should also fix this for 1.2.1. What do you
> >>>>>>
> >>>>>> think?
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Regards,
> >>>>>>>>>>>>>>>>>>>> Timo
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Am 29/03/17 um 11:30 schrieb Robert Metzger:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Hi Haohui,
> >>>>>>>>>>>>>>>>>>>>> I agree that we should fix the parallelism issue.
> >>>>>>
> >>>>>> Otherwise,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> 1.2.1
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> release would introduce a new bug.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On Tue, Mar 28, 2017 at 11:59 PM, Haohui Mai <
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> ricetons@gmail.com>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> -1 (non-binding)
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> We recently found out that all jobs submitted via UI
> >>>
> >>> will
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> have a
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> parallelism of 1, potentially due to FLINK-5808.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Filed FLINK-6209 to track it.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> ~Haohui
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Mon, Mar 27, 2017 at 2:59 AM Chesnay Schepler <
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> chesnay@apache.org>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> If possible I would like to include FLINK-6183 &
> >>>>>>
> >>>>>> FLINK-6184
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> well.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> They fix 2 metric-related issues that could arise
> >>>
> >>> when a
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Task is
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> cancelled very early. (like, right away)
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> FLINK-6183 fixes a memory leak where the
> >>>
> >>> TaskMetricGroup
> >>>>>>>>>>
> >>>>>>>>>> was
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> never closed
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> FLINK-6184 fixes a NullPointerExceptions in the
> >>>>>>>>>>>>>>>>>>>>>>> buffer
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> metrics
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> PR here: https://github.com/apache/flink/pull/3611
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> On 26.03.2017 12:35, Aljoscha Krettek wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> I opened a PR for FLINK-6188:
> >>>>>>
> >>>>>> https://github.com/apache/
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> flink/pull/3616
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> <https://github.com/apache/flink/pull/3616>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> This improves the previously very sparse test
> >>>
> >>> coverage
> >>>>>>
> >>>>>> for
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> timestamp/watermark assigners and fixes the bug.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> On 25 Mar 2017, at 10:22, Ufuk Celebi <
> >>>
> >>> uce@apache.org>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> I agree with Aljoscha.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> -1 because of FLINK-6188
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> On Sat, Mar 25, 2017 at 9:38 AM, Aljoscha
> Krettek <
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> aljoscha@apache.org>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> I filed this issue, which was observed by a
> user:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6188
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>> I think that’s blocking for 1.2.1.
> >>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> On 24 Mar 2017, at 18:57, Ufuk Celebi <
> >>>>>>
> >>>>>> uce@apache.org>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> RC1 doesn't contain Stefan's backport for the
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Asynchronous
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> snapshots
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> for heap-based keyed state that has been
> merged.
> >>>>>>
> >>>>>> Should
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> create
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> RC2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> with that fix since the voting period only
> starts
> >>>
> >>> on
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Monday?
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> I think
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> it would only mean rerunning the scripts on
> your
> >>>>>>
> >>>>>> side,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> right?
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> – Ufuk
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Mar 24, 2017 at 3:05 PM, Robert
> Metzger <
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> rmetzger@apache.org>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Dear Flink community,
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Please vote on releasing the following
> candidate
> >>>
> >>> as
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Apache
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Flink
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> version 1.2
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> .1.
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> The commit to be voted on:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *732e55bd* (*
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> http://git-wip-us.apache.org/repos/asf/flink/commit/
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> 732e55bd
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> <http://git-wip-us.apache.org/
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> repos/asf/flink/commit/732e55b
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> d>*)
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Branch:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> release-1.2.1-rc1
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> The release artifacts to be voted on can be
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> found
> >>>>>>
> >>>>>> at:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> *http://people.apache.org/~
> >>>>>>
> >>>>>> rmetzger/flink-1.2.1-rc1/
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> <http://people.apache.org/~
> >>>>>>
> >>>>>> rmetzger/flink-1.2.1-rc1/
> >>>>>>>>>>>
> >>>>>>>>>>> *
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> The release artifacts are signed with the key
> >>>
> >>> with
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> fingerprint
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> D9839159:
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> http://www.apache.org/dist/flink/KEYS
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>>>> The staging repository for this release can be
> >>>
> >>> found
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> at:
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> https://repository.apache.org/
>
Re: [VOTE] Release Apache Flink 1.2.1 (RC1)
Posted by Ufuk Celebi <uc...@apache.org>.
@Stefan: What's the state with the RocksDB fixes? I would be +1 to do this.
On Tue, Apr 4, 2017 at 6:05 PM, Chesnay Schepler <ch...@apache.org> wrote:
> Yes, aljoscha already opened one against master:
> https://github.com/apache/flink/pull/3670
>
> On 04.04.2017 17:57, Ted Yu wrote:
>>
>> Should the commits be reverted from master branch as well ?
>>
>> On Tue, Apr 4, 2017 at 4:59 AM, Aljoscha Krettek <al...@apache.org>
>> wrote:
>>
>>> The commits around FLINK-5808 have been reverted on release-1.2.
>>>
>>>> On 4. Apr 2017, at 12:16, Stefan Richter <s....@data-artisans.com>
>>>
>>> wrote:
>>>>
>>>> I have created a custom build of RocksDB 4.11.2 that fixes a significant
>>>
>>> performance problem with append operations. I think this should
>>> definitely
>>> be part of the 1.2.1 release because this is already blocking some users.
>>> What is missing is uploading the jar to maven central and a testing run,
>>> e.g. with some misbehaved job that has large state.
>>>>
>>>>
>>>>> Am 04.04.2017 um 11:57 schrieb Robert Metzger <rm...@apache.org>:
>>>>>
>>>>> Thank you for opening a PR for this.
>>>>>
>>>>> Chesnay, do you need more reviews for the metrics changes / backports?
>>>>>
>>>>> Are there any other release blockers for 1.2.1, or are we good to go?
>>>>>
>>>>> On Mon, Apr 3, 2017 at 6:48 PM, Aljoscha Krettek <al...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> I created a PR for the revert: https://github.com/apache/
>>>
>>> flink/pull/3664
>>>>>>>
>>>>>>> On 3. Apr 2017, at 18:32, Stephan Ewen <se...@apache.org> wrote:
>>>>>>>
>>>>>>> +1 for options (1), but also invest the time to fix it properly for
>>>
>>> 1.2.2
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Apr 3, 2017 at 9:10 AM, Kostas Kloudas <
>>>>>>
>>>>>> k.kloudas@data-artisans.com>
>>>>>>>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> +1 for 1
>>>>>>>>
>>>>>>>>> On Apr 3, 2017, at 5:52 PM, Till Rohrmann <tr...@apache.org>
>>>>>>
>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> +1 for option 1)
>>>>>>>>>
>>>>>>>>> On Mon, Apr 3, 2017 at 5:48 PM, Fabian Hueske <fh...@gmail.com>
>>>>>>
>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> +1 to option 1)
>>>>>>>>>>
>>>>>>>>>> 2017-04-03 16:57 GMT+02:00 Ted Yu <yu...@gmail.com>:
>>>>>>>>>>
>>>>>>>>>>> Looks like #1 is better - 1.2.1 would be at least as stable as
>>>
>>> 1.2.0
>>>>>>>>>>>
>>>>>>>>>>> Cheers
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Apr 3, 2017 at 7:39 AM, Aljoscha Krettek <
>>>>>>
>>>>>> aljoscha@apache.org>
>>>>>>>>>>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Just so we’re all on the same page. ;-)
>>>>>>>>>>>>
>>>>>>>>>>>> There was https://issues.apache.org/jira/browse/FLINK-5808 which
>>>>>>
>>>>>> was
>>>>>>>>
>>>>>>>> a
>>>>>>>>>>>>
>>>>>>>>>>>> bug that we initially discovered in Flink 1.2 which was/is about
>>>>>>>>>>
>>>>>>>>>> missing
>>>>>>>>>>>>
>>>>>>>>>>>> verification for the correctness of the combination of
>>>
>>> parallelism
>>>>>>
>>>>>> and
>>>>>>>>>>>>
>>>>>>>>>>>> max-parallelism. Due to lacking test coverage this introduced
>>>>>>>>>>>> two
>>>>>>
>>>>>> more
>>>>>>>>>>>
>>>>>>>>>>> bugs:
>>>>>>>>>>>>
>>>>>>>>>>>> - https://issues.apache.org/jira/browse/FLINK-6188: Some
>>>>>>>>>>>> setParallelism() methods can't cope with default parallelism
>>>>>>>>>>>> - https://issues.apache.org/jira/browse/FLINK-6209:
>>>>>>>>>>>> StreamPlanEnvironment always has a parallelism of 1
>>>>>>>>>>>>
>>>>>>>>>>>> IMHO, the options are:
>>>>>>>>>>>> 1) revert the changes made for FLINK-5808 on the release-1.2
>>>
>>> branch
>>>>>>>>>>
>>>>>>>>>> and
>>>>>>>>>>>>
>>>>>>>>>>>> live with the bug still being present
>>>>>>>>>>>> 2) put in more work to fix FLINK-5808 which requires fixing some
>>>>>>>>>>>
>>>>>>>>>>> problems
>>>>>>>>>>>>
>>>>>>>>>>>> that have existed for a long time with how the parallelism is
>>>
>>> set in
>>>>>>>>>>>>
>>>>>>>>>>>> streaming programs
>>>>>>>>>>>>
>>>>>>>>>>>> Best,
>>>>>>>>>>>> Aljoscha
>>>>>>>>>>>>
>>>>>>>>>>>>> On 31. Mar 2017, at 21:34, Robert Metzger <rm...@apache.org>
>>>>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> I don't know what is best to do, but I think releasing 1.2.1
>>>
>>> with
>>>>>>>>>>>>>
>>>>>>>>>>>>> potentially more bugs than 1.2.0 is not a good option.
>>>>>>>>>>>>> I suspect a good workaround for FLINK-6188
>>>>>>>>>>>>> <https://issues.apache.org/jira/browse/FLINK-6188> is setting
>>>
>>> the
>>>>>>>>>>>>>
>>>>>>>>>>>>> parallelism manually for operators that can't cope with the
>>>
>>> default
>>>>>>>>>>
>>>>>>>>>> -1
>>>>>>>>>>>>>
>>>>>>>>>>>>> parallelism.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Mar 31, 2017 at 9:06 PM, Aljoscha Krettek <
>>>>>>>>>>
>>>>>>>>>> aljoscha@apache.org
>>>>>>>>>>>>>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> You mean reverting the changes around FLINK-5808 [1]? This is
>>>
>>> what
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> introduced the follow-up FLINK-6188 [2].
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/FLINK-5808
>>>>>>>>>>>>>> [2]https://issues.apache.org/jira/browse/FLINK-6188
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Fri, Mar 31, 2017, at 19:10, Robert Metzger wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I think reverting FLINK-6188 for the 1.2 branch might be a
>>>
>>> good
>>>>>>>>>>
>>>>>>>>>> idea.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> FLINK-6188 introduced two new bugs, so undoing the FLINK-6188
>>>
>>> fix
>>>>>>>>>>>
>>>>>>>>>>> will
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> lead
>>>>>>>>>>>>>>> only to one known bug in 1.2.1, instead of an uncertain
>>>
>>> number of
>>>>>>>>>>>>
>>>>>>>>>>>> issues.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> So 1.2.1 is not going to be worse than 1.2.0
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The fix will hopefully make it into 1.2.2 then.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Any other thoughts on this?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Fri, Mar 31, 2017 at 6:46 PM, Fabian Hueske <
>>>>>>
>>>>>> fhueske@gmail.com>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I merged the fix for FLINK-6044 to the release-1.2 and
>>>>>>
>>>>>> release-1.1
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> branch.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 2017-03-31 15:02 GMT+02:00 Fabian Hueske <fhueske@gmail.com
>>>>
>>>> :
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> We should also backport the fix for FLINK-6044 to Flink
>>>
>>> 1.2.1.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I'll take care of that.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 2017-03-30 18:50 GMT+02:00 Aljoscha Krettek <
>>>>>>
>>>>>> aljoscha@apache.org
>>>>>>>>>>>
>>>>>>>>>>> :
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6188 turns
>>>
>>> out to
>>>>>>>>>>
>>>>>>>>>> be
>>>>>>>>>>>
>>>>>>>>>>> a
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> bit
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> more involved, see my comments on the PR:
>>>>>>>>>>>>>>>>>> https://github.com/apache/flink/pull/3616.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> As I said there, maybe we should revert the commits
>>>
>>> regarding
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> parallelism/max-parallelism changes and release and then
>>>
>>> fix
>>>>>>
>>>>>> it
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> later.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2017, at 23:08, Aljoscha Krettek wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I commented on FLINK-6214: I think it's working as
>>>
>>> intended,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> although
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> could fix the javadoc/doc.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2017, at 17:35, Timo Walther wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> A user reported that all tumbling and slinding window
>>>>>>>>>>
>>>>>>>>>> assigners
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> contain
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> a pretty obvious bug about offsets.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6214
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I think we should also fix this for 1.2.1. What do you
>>>>>>
>>>>>> think?
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>> Timo
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Am 29/03/17 um 11:30 schrieb Robert Metzger:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Hi Haohui,
>>>>>>>>>>>>>>>>>>>>> I agree that we should fix the parallelism issue.
>>>>>>
>>>>>> Otherwise,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 1.2.1
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> release would introduce a new bug.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Tue, Mar 28, 2017 at 11:59 PM, Haohui Mai <
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ricetons@gmail.com>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> -1 (non-binding)
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> We recently found out that all jobs submitted via UI
>>>
>>> will
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> have a
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> parallelism of 1, potentially due to FLINK-5808.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Filed FLINK-6209 to track it.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> ~Haohui
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Mon, Mar 27, 2017 at 2:59 AM Chesnay Schepler <
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> chesnay@apache.org>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> If possible I would like to include FLINK-6183 &
>>>>>>
>>>>>> FLINK-6184
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> well.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> They fix 2 metric-related issues that could arise
>>>
>>> when a
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Task is
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> cancelled very early. (like, right away)
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> FLINK-6183 fixes a memory leak where the
>>>
>>> TaskMetricGroup
>>>>>>>>>>
>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> never closed
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> FLINK-6184 fixes a NullPointerExceptions in the
>>>>>>>>>>>>>>>>>>>>>>> buffer
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> metrics
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> PR here: https://github.com/apache/flink/pull/3611
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On 26.03.2017 12:35, Aljoscha Krettek wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> I opened a PR for FLINK-6188:
>>>>>>
>>>>>> https://github.com/apache/
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> flink/pull/3616
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> <https://github.com/apache/flink/pull/3616>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> This improves the previously very sparse test
>>>
>>> coverage
>>>>>>
>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> timestamp/watermark assigners and fixes the bug.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> On 25 Mar 2017, at 10:22, Ufuk Celebi <
>>>
>>> uce@apache.org>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> I agree with Aljoscha.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> -1 because of FLINK-6188
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> On Sat, Mar 25, 2017 at 9:38 AM, Aljoscha Krettek <
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> aljoscha@apache.org>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> I filed this issue, which was observed by a user:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6188
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> I think that’s blocking for 1.2.1.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> On 24 Mar 2017, at 18:57, Ufuk Celebi <
>>>>>>
>>>>>> uce@apache.org>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> RC1 doesn't contain Stefan's backport for the
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Asynchronous
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> snapshots
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> for heap-based keyed state that has been merged.
>>>>>>
>>>>>> Should
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> RC2
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> with that fix since the voting period only starts
>>>
>>> on
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Monday?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I think
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> it would only mean rerunning the scripts on your
>>>>>>
>>>>>> side,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> right?
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> – Ufuk
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Mar 24, 2017 at 3:05 PM, Robert Metzger <
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> rmetzger@apache.org>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dear Flink community,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please vote on releasing the following candidate
>>>
>>> as
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Apache
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Flink
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> version 1.2
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> .1.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> The commit to be voted on:
>>>>>>>>>>>>>>>>>>>>>>>>>>>> *732e55bd* (*
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> http://git-wip-us.apache.org/repos/asf/flink/commit/
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 732e55bd
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> <http://git-wip-us.apache.org/
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> repos/asf/flink/commit/732e55b
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> d>*)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Branch:
>>>>>>>>>>>>>>>>>>>>>>>>>>>> release-1.2.1-rc1
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> The release artifacts to be voted on can be
>>>>>>>>>>>>>>>>>>>>>>>>>>>> found
>>>>>>
>>>>>> at:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> *http://people.apache.org/~
>>>>>>
>>>>>> rmetzger/flink-1.2.1-rc1/
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> <http://people.apache.org/~
>>>>>>
>>>>>> rmetzger/flink-1.2.1-rc1/
>>>>>>>>>>>
>>>>>>>>>>> *
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> The release artifacts are signed with the key
>>>
>>> with
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> fingerprint
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> D9839159:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://www.apache.org/dist/flink/KEYS
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> The staging repository for this release can be
>>>
>>> found
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> at:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> https://repository.apache.org/
Re: [VOTE] Release Apache Flink 1.2.1 (RC1)
Posted by Chesnay Schepler <ch...@apache.org>.
Yes, aljoscha already opened one against master:
https://github.com/apache/flink/pull/3670
On 04.04.2017 17:57, Ted Yu wrote:
> Should the commits be reverted from master branch as well ?
>
> On Tue, Apr 4, 2017 at 4:59 AM, Aljoscha Krettek <al...@apache.org>
> wrote:
>
>> The commits around FLINK-5808 have been reverted on release-1.2.
>>
>>> On 4. Apr 2017, at 12:16, Stefan Richter <s....@data-artisans.com>
>> wrote:
>>> I have created a custom build of RocksDB 4.11.2 that fixes a significant
>> performance problem with append operations. I think this should definitely
>> be part of the 1.2.1 release because this is already blocking some users.
>> What is missing is uploading the jar to maven central and a testing run,
>> e.g. with some misbehaved job that has large state.
>>>
>>>> Am 04.04.2017 um 11:57 schrieb Robert Metzger <rm...@apache.org>:
>>>>
>>>> Thank you for opening a PR for this.
>>>>
>>>> Chesnay, do you need more reviews for the metrics changes / backports?
>>>>
>>>> Are there any other release blockers for 1.2.1, or are we good to go?
>>>>
>>>> On Mon, Apr 3, 2017 at 6:48 PM, Aljoscha Krettek <al...@apache.org>
>>>> wrote:
>>>>
>>>>> I created a PR for the revert: https://github.com/apache/
>> flink/pull/3664
>>>>>> On 3. Apr 2017, at 18:32, Stephan Ewen <se...@apache.org> wrote:
>>>>>>
>>>>>> +1 for options (1), but also invest the time to fix it properly for
>> 1.2.2
>>>>>>
>>>>>> On Mon, Apr 3, 2017 at 9:10 AM, Kostas Kloudas <
>>>>> k.kloudas@data-artisans.com>
>>>>>> wrote:
>>>>>>
>>>>>>> +1 for 1
>>>>>>>
>>>>>>>> On Apr 3, 2017, at 5:52 PM, Till Rohrmann <tr...@apache.org>
>>>>> wrote:
>>>>>>>> +1 for option 1)
>>>>>>>>
>>>>>>>> On Mon, Apr 3, 2017 at 5:48 PM, Fabian Hueske <fh...@gmail.com>
>>>>> wrote:
>>>>>>>>> +1 to option 1)
>>>>>>>>>
>>>>>>>>> 2017-04-03 16:57 GMT+02:00 Ted Yu <yu...@gmail.com>:
>>>>>>>>>
>>>>>>>>>> Looks like #1 is better - 1.2.1 would be at least as stable as
>> 1.2.0
>>>>>>>>>> Cheers
>>>>>>>>>>
>>>>>>>>>> On Mon, Apr 3, 2017 at 7:39 AM, Aljoscha Krettek <
>>>>> aljoscha@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Just so we\u2019re all on the same page. ;-)
>>>>>>>>>>>
>>>>>>>>>>> There was https://issues.apache.org/jira/browse/FLINK-5808 which
>>>>> was
>>>>>>> a
>>>>>>>>>>> bug that we initially discovered in Flink 1.2 which was/is about
>>>>>>>>> missing
>>>>>>>>>>> verification for the correctness of the combination of
>> parallelism
>>>>> and
>>>>>>>>>>> max-parallelism. Due to lacking test coverage this introduced two
>>>>> more
>>>>>>>>>> bugs:
>>>>>>>>>>> - https://issues.apache.org/jira/browse/FLINK-6188: Some
>>>>>>>>>>> setParallelism() methods can't cope with default parallelism
>>>>>>>>>>> - https://issues.apache.org/jira/browse/FLINK-6209:
>>>>>>>>>>> StreamPlanEnvironment always has a parallelism of 1
>>>>>>>>>>>
>>>>>>>>>>> IMHO, the options are:
>>>>>>>>>>> 1) revert the changes made for FLINK-5808 on the release-1.2
>> branch
>>>>>>>>> and
>>>>>>>>>>> live with the bug still being present
>>>>>>>>>>> 2) put in more work to fix FLINK-5808 which requires fixing some
>>>>>>>>>> problems
>>>>>>>>>>> that have existed for a long time with how the parallelism is
>> set in
>>>>>>>>>>> streaming programs
>>>>>>>>>>>
>>>>>>>>>>> Best,
>>>>>>>>>>> Aljoscha
>>>>>>>>>>>
>>>>>>>>>>>> On 31. Mar 2017, at 21:34, Robert Metzger <rm...@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>>>> I don't know what is best to do, but I think releasing 1.2.1
>> with
>>>>>>>>>>>> potentially more bugs than 1.2.0 is not a good option.
>>>>>>>>>>>> I suspect a good workaround for FLINK-6188
>>>>>>>>>>>> <https://issues.apache.org/jira/browse/FLINK-6188> is setting
>> the
>>>>>>>>>>>> parallelism manually for operators that can't cope with the
>> default
>>>>>>>>> -1
>>>>>>>>>>>> parallelism.
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Mar 31, 2017 at 9:06 PM, Aljoscha Krettek <
>>>>>>>>> aljoscha@apache.org
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> You mean reverting the changes around FLINK-5808 [1]? This is
>> what
>>>>>>>>>>>>> introduced the follow-up FLINK-6188 [2].
>>>>>>>>>>>>>
>>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/FLINK-5808
>>>>>>>>>>>>> [2]https://issues.apache.org/jira/browse/FLINK-6188
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Mar 31, 2017, at 19:10, Robert Metzger wrote:
>>>>>>>>>>>>>> I think reverting FLINK-6188 for the 1.2 branch might be a
>> good
>>>>>>>>> idea.
>>>>>>>>>>>>>> FLINK-6188 introduced two new bugs, so undoing the FLINK-6188
>> fix
>>>>>>>>>> will
>>>>>>>>>>>>>> lead
>>>>>>>>>>>>>> only to one known bug in 1.2.1, instead of an uncertain
>> number of
>>>>>>>>>>> issues.
>>>>>>>>>>>>>> So 1.2.1 is not going to be worse than 1.2.0
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The fix will hopefully make it into 1.2.2 then.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Any other thoughts on this?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Fri, Mar 31, 2017 at 6:46 PM, Fabian Hueske <
>>>>> fhueske@gmail.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> I merged the fix for FLINK-6044 to the release-1.2 and
>>>>> release-1.1
>>>>>>>>>>>>> branch.
>>>>>>>>>>>>>>> 2017-03-31 15:02 GMT+02:00 Fabian Hueske <fhueske@gmail.com
>>> :
>>>>>>>>>>>>>>>> We should also backport the fix for FLINK-6044 to Flink
>> 1.2.1.
>>>>>>>>>>>>>>>> I'll take care of that.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 2017-03-30 18:50 GMT+02:00 Aljoscha Krettek <
>>>>> aljoscha@apache.org
>>>>>>>>>> :
>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6188 turns
>> out to
>>>>>>>>> be
>>>>>>>>>> a
>>>>>>>>>>>>> bit
>>>>>>>>>>>>>>>>> more involved, see my comments on the PR:
>>>>>>>>>>>>>>>>> https://github.com/apache/flink/pull/3616.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> As I said there, maybe we should revert the commits
>> regarding
>>>>>>>>>>>>>>>>> parallelism/max-parallelism changes and release and then
>> fix
>>>>> it
>>>>>>>>>>>>> later.
>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2017, at 23:08, Aljoscha Krettek wrote:
>>>>>>>>>>>>>>>>>> I commented on FLINK-6214: I think it's working as
>> intended,
>>>>>>>>>>>>> although
>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>> could fix the javadoc/doc.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Wed, Mar 29, 2017, at 17:35, Timo Walther wrote:
>>>>>>>>>>>>>>>>>>> A user reported that all tumbling and slinding window
>>>>>>>>> assigners
>>>>>>>>>>>>>>>>> contain
>>>>>>>>>>>>>>>>>>> a pretty obvious bug about offsets.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6214
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I think we should also fix this for 1.2.1. What do you
>>>>> think?
>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>> Timo
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Am 29/03/17 um 11:30 schrieb Robert Metzger:
>>>>>>>>>>>>>>>>>>>> Hi Haohui,
>>>>>>>>>>>>>>>>>>>> I agree that we should fix the parallelism issue.
>>>>> Otherwise,
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> 1.2.1
>>>>>>>>>>>>>>>>>>>> release would introduce a new bug.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Tue, Mar 28, 2017 at 11:59 PM, Haohui Mai <
>>>>>>>>>>>>> ricetons@gmail.com>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>> -1 (non-binding)
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> We recently found out that all jobs submitted via UI
>> will
>>>>>>>>>>>>> have a
>>>>>>>>>>>>>>>>>>>>> parallelism of 1, potentially due to FLINK-5808.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Filed FLINK-6209 to track it.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> ~Haohui
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Mon, Mar 27, 2017 at 2:59 AM Chesnay Schepler <
>>>>>>>>>>>>>>>>> chesnay@apache.org>
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> If possible I would like to include FLINK-6183 &
>>>>> FLINK-6184
>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>> well.
>>>>>>>>>>>>>>>>>>>>>> They fix 2 metric-related issues that could arise
>> when a
>>>>>>>>>>>>> Task is
>>>>>>>>>>>>>>>>>>>>>> cancelled very early. (like, right away)
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> FLINK-6183 fixes a memory leak where the
>> TaskMetricGroup
>>>>>>>>> was
>>>>>>>>>>>>>>>>> never closed
>>>>>>>>>>>>>>>>>>>>>> FLINK-6184 fixes a NullPointerExceptions in the buffer
>>>>>>>>>>>>> metrics
>>>>>>>>>>>>>>>>>>>>>> PR here: https://github.com/apache/flink/pull/3611
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On 26.03.2017 12:35, Aljoscha Krettek wrote:
>>>>>>>>>>>>>>>>>>>>>>> I opened a PR for FLINK-6188:
>>>>> https://github.com/apache/
>>>>>>>>>>>>>>>>>>>>> flink/pull/3616
>>>>>>>>>>>>>>>>>>>>>> <https://github.com/apache/flink/pull/3616>
>>>>>>>>>>>>>>>>>>>>>>> This improves the previously very sparse test
>> coverage
>>>>> for
>>>>>>>>>>>>>>>>>>>>>> timestamp/watermark assigners and fixes the bug.
>>>>>>>>>>>>>>>>>>>>>>>> On 25 Mar 2017, at 10:22, Ufuk Celebi <
>> uce@apache.org>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>> I agree with Aljoscha.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> -1 because of FLINK-6188
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On Sat, Mar 25, 2017 at 9:38 AM, Aljoscha Krettek <
>>>>>>>>>>>>>>>>>>>>> aljoscha@apache.org>
>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>> I filed this issue, which was observed by a user:
>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6188
>>>>>>>>>>>>>>>>>>>>>>>>> I think that\u2019s blocking for 1.2.1.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> On 24 Mar 2017, at 18:57, Ufuk Celebi <
>>>>> uce@apache.org>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>> RC1 doesn't contain Stefan's backport for the
>>>>>>>>>>>>> Asynchronous
>>>>>>>>>>>>>>>>> snapshots
>>>>>>>>>>>>>>>>>>>>>>>>>> for heap-based keyed state that has been merged.
>>>>> Should
>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>>>>>> RC2
>>>>>>>>>>>>>>>>>>>>>>>>>> with that fix since the voting period only starts
>> on
>>>>>>>>>>>>> Monday?
>>>>>>>>>>>>>>>>> I think
>>>>>>>>>>>>>>>>>>>>>>>>>> it would only mean rerunning the scripts on your
>>>>> side,
>>>>>>>>>>>>>>> right?
>>>>>>>>>>>>>>>>>>>>>>>>>> \u2013 Ufuk
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Mar 24, 2017 at 3:05 PM, Robert Metzger <
>>>>>>>>>>>>>>>>>>>>> rmetzger@apache.org>
>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>> Dear Flink community,
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Please vote on releasing the following candidate
>> as
>>>>>>>>>>>>> Apache
>>>>>>>>>>>>>>>>> Flink
>>>>>>>>>>>>>>>>>>>>>> version 1.2
>>>>>>>>>>>>>>>>>>>>>>>>>>> .1.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> The commit to be voted on:
>>>>>>>>>>>>>>>>>>>>>>>>>>> *732e55bd* (*
>>>>>>>>>>>>>>>>>>>>>> http://git-wip-us.apache.org/repos/asf/flink/commit/
>>>>>>>>>>>>> 732e55bd
>>>>>>>>>>>>>>>>>>>>>>>>>>> <http://git-wip-us.apache.org/
>>>>>>>>>>>>>>> repos/asf/flink/commit/732e55b
>>>>>>>>>>>>>>>>> d>*)
>>>>>>>>>>>>>>>>>>>>>>>>>>> Branch:
>>>>>>>>>>>>>>>>>>>>>>>>>>> release-1.2.1-rc1
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> The release artifacts to be voted on can be found
>>>>> at:
>>>>>>>>>>>>>>>>>>>>>>>>>>> *http://people.apache.org/~
>>>>> rmetzger/flink-1.2.1-rc1/
>>>>>>>>>>>>>>>>>>>>>>>>>>> <http://people.apache.org/~
>>>>> rmetzger/flink-1.2.1-rc1/
>>>>>>>>>> *
>>>>>>>>>>>>>>>>>>>>>>>>>>> The release artifacts are signed with the key
>> with
>>>>>>>>>>>>>>>>> fingerprint
>>>>>>>>>>>>>>>>>>>>>> D9839159:
>>>>>>>>>>>>>>>>>>>>>>>>>>> http://www.apache.org/dist/flink/KEYS
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> The staging repository for this release can be
>> found
>>>>>>>>>>>>> at:
>>>>>>>>>>>>>>>>>>>>>> https://repository.apache.org/
>>>>>>>>>>>>> content/repositories/orgapache
>>>>>>>>>>>>>>>>> flink-1116
>>>>>>>>>>>>>>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>>>>> -
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> The vote ends on Wednesday, March 29, 2017, 3pm
>> CET.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> [ ] +1 Release this package as Apache Flink 1.2.1
>>>>>>>>>>>>>>>>>>>>>>>>>>> [ ] -1 Do not release this package, because ...
>>>>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>
>>>>>
>>
Re: [VOTE] Release Apache Flink 1.2.1 (RC1)
Posted by Ted Yu <yu...@gmail.com>.
Should the commits be reverted from master branch as well ?
On Tue, Apr 4, 2017 at 4:59 AM, Aljoscha Krettek <al...@apache.org>
wrote:
> The commits around FLINK-5808 have been reverted on release-1.2.
>
> > On 4. Apr 2017, at 12:16, Stefan Richter <s....@data-artisans.com>
> wrote:
> >
> > I have created a custom build of RocksDB 4.11.2 that fixes a significant
> performance problem with append operations. I think this should definitely
> be part of the 1.2.1 release because this is already blocking some users.
> What is missing is uploading the jar to maven central and a testing run,
> e.g. with some misbehaved job that has large state.
> >
> >
> >> Am 04.04.2017 um 11:57 schrieb Robert Metzger <rm...@apache.org>:
> >>
> >> Thank you for opening a PR for this.
> >>
> >> Chesnay, do you need more reviews for the metrics changes / backports?
> >>
> >> Are there any other release blockers for 1.2.1, or are we good to go?
> >>
> >> On Mon, Apr 3, 2017 at 6:48 PM, Aljoscha Krettek <al...@apache.org>
> >> wrote:
> >>
> >>> I created a PR for the revert: https://github.com/apache/
> flink/pull/3664
> >>>
> >>>> On 3. Apr 2017, at 18:32, Stephan Ewen <se...@apache.org> wrote:
> >>>>
> >>>> +1 for options (1), but also invest the time to fix it properly for
> 1.2.2
> >>>>
> >>>>
> >>>> On Mon, Apr 3, 2017 at 9:10 AM, Kostas Kloudas <
> >>> k.kloudas@data-artisans.com>
> >>>> wrote:
> >>>>
> >>>>> +1 for 1
> >>>>>
> >>>>>> On Apr 3, 2017, at 5:52 PM, Till Rohrmann <tr...@apache.org>
> >>> wrote:
> >>>>>>
> >>>>>> +1 for option 1)
> >>>>>>
> >>>>>> On Mon, Apr 3, 2017 at 5:48 PM, Fabian Hueske <fh...@gmail.com>
> >>> wrote:
> >>>>>>
> >>>>>>> +1 to option 1)
> >>>>>>>
> >>>>>>> 2017-04-03 16:57 GMT+02:00 Ted Yu <yu...@gmail.com>:
> >>>>>>>
> >>>>>>>> Looks like #1 is better - 1.2.1 would be at least as stable as
> 1.2.0
> >>>>>>>>
> >>>>>>>> Cheers
> >>>>>>>>
> >>>>>>>> On Mon, Apr 3, 2017 at 7:39 AM, Aljoscha Krettek <
> >>> aljoscha@apache.org>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Just so we’re all on the same page. ;-)
> >>>>>>>>>
> >>>>>>>>> There was https://issues.apache.org/jira/browse/FLINK-5808 which
> >>> was
> >>>>> a
> >>>>>>>>> bug that we initially discovered in Flink 1.2 which was/is about
> >>>>>>> missing
> >>>>>>>>> verification for the correctness of the combination of
> parallelism
> >>> and
> >>>>>>>>> max-parallelism. Due to lacking test coverage this introduced two
> >>> more
> >>>>>>>> bugs:
> >>>>>>>>> - https://issues.apache.org/jira/browse/FLINK-6188: Some
> >>>>>>>>> setParallelism() methods can't cope with default parallelism
> >>>>>>>>> - https://issues.apache.org/jira/browse/FLINK-6209:
> >>>>>>>>> StreamPlanEnvironment always has a parallelism of 1
> >>>>>>>>>
> >>>>>>>>> IMHO, the options are:
> >>>>>>>>> 1) revert the changes made for FLINK-5808 on the release-1.2
> branch
> >>>>>>> and
> >>>>>>>>> live with the bug still being present
> >>>>>>>>> 2) put in more work to fix FLINK-5808 which requires fixing some
> >>>>>>>> problems
> >>>>>>>>> that have existed for a long time with how the parallelism is
> set in
> >>>>>>>>> streaming programs
> >>>>>>>>>
> >>>>>>>>> Best,
> >>>>>>>>> Aljoscha
> >>>>>>>>>
> >>>>>>>>>> On 31. Mar 2017, at 21:34, Robert Metzger <rm...@apache.org>
> >>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> I don't know what is best to do, but I think releasing 1.2.1
> with
> >>>>>>>>>> potentially more bugs than 1.2.0 is not a good option.
> >>>>>>>>>> I suspect a good workaround for FLINK-6188
> >>>>>>>>>> <https://issues.apache.org/jira/browse/FLINK-6188> is setting
> the
> >>>>>>>>>> parallelism manually for operators that can't cope with the
> default
> >>>>>>> -1
> >>>>>>>>>> parallelism.
> >>>>>>>>>>
> >>>>>>>>>> On Fri, Mar 31, 2017 at 9:06 PM, Aljoscha Krettek <
> >>>>>>> aljoscha@apache.org
> >>>>>>>>>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> You mean reverting the changes around FLINK-5808 [1]? This is
> what
> >>>>>>>>>>> introduced the follow-up FLINK-6188 [2].
> >>>>>>>>>>>
> >>>>>>>>>>> [1] https://issues.apache.org/jira/browse/FLINK-5808
> >>>>>>>>>>> [2]https://issues.apache.org/jira/browse/FLINK-6188
> >>>>>>>>>>>
> >>>>>>>>>>> On Fri, Mar 31, 2017, at 19:10, Robert Metzger wrote:
> >>>>>>>>>>>> I think reverting FLINK-6188 for the 1.2 branch might be a
> good
> >>>>>>> idea.
> >>>>>>>>>>>> FLINK-6188 introduced two new bugs, so undoing the FLINK-6188
> fix
> >>>>>>>> will
> >>>>>>>>>>>> lead
> >>>>>>>>>>>> only to one known bug in 1.2.1, instead of an uncertain
> number of
> >>>>>>>>> issues.
> >>>>>>>>>>>> So 1.2.1 is not going to be worse than 1.2.0
> >>>>>>>>>>>>
> >>>>>>>>>>>> The fix will hopefully make it into 1.2.2 then.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Any other thoughts on this?
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Fri, Mar 31, 2017 at 6:46 PM, Fabian Hueske <
> >>> fhueske@gmail.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> I merged the fix for FLINK-6044 to the release-1.2 and
> >>> release-1.1
> >>>>>>>>>>> branch.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 2017-03-31 15:02 GMT+02:00 Fabian Hueske <fhueske@gmail.com
> >:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> We should also backport the fix for FLINK-6044 to Flink
> 1.2.1.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I'll take care of that.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> 2017-03-30 18:50 GMT+02:00 Aljoscha Krettek <
> >>> aljoscha@apache.org
> >>>>>>>> :
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6188 turns
> out to
> >>>>>>> be
> >>>>>>>> a
> >>>>>>>>>>> bit
> >>>>>>>>>>>>>>> more involved, see my comments on the PR:
> >>>>>>>>>>>>>>> https://github.com/apache/flink/pull/3616.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> As I said there, maybe we should revert the commits
> regarding
> >>>>>>>>>>>>>>> parallelism/max-parallelism changes and release and then
> fix
> >>> it
> >>>>>>>>>>> later.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Wed, Mar 29, 2017, at 23:08, Aljoscha Krettek wrote:
> >>>>>>>>>>>>>>>> I commented on FLINK-6214: I think it's working as
> intended,
> >>>>>>>>>>> although
> >>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>> could fix the javadoc/doc.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Wed, Mar 29, 2017, at 17:35, Timo Walther wrote:
> >>>>>>>>>>>>>>>>> A user reported that all tumbling and slinding window
> >>>>>>> assigners
> >>>>>>>>>>>>>>> contain
> >>>>>>>>>>>>>>>>> a pretty obvious bug about offsets.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6214
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I think we should also fix this for 1.2.1. What do you
> >>> think?
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Regards,
> >>>>>>>>>>>>>>>>> Timo
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Am 29/03/17 um 11:30 schrieb Robert Metzger:
> >>>>>>>>>>>>>>>>>> Hi Haohui,
> >>>>>>>>>>>>>>>>>> I agree that we should fix the parallelism issue.
> >>> Otherwise,
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>> 1.2.1
> >>>>>>>>>>>>>>>>>> release would introduce a new bug.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Tue, Mar 28, 2017 at 11:59 PM, Haohui Mai <
> >>>>>>>>>>> ricetons@gmail.com>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> -1 (non-binding)
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> We recently found out that all jobs submitted via UI
> will
> >>>>>>>>>>> have a
> >>>>>>>>>>>>>>>>>>> parallelism of 1, potentially due to FLINK-5808.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Filed FLINK-6209 to track it.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> ~Haohui
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Mon, Mar 27, 2017 at 2:59 AM Chesnay Schepler <
> >>>>>>>>>>>>>>> chesnay@apache.org>
> >>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> If possible I would like to include FLINK-6183 &
> >>> FLINK-6184
> >>>>>>>>>>> as
> >>>>>>>>>>>>>>> well.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> They fix 2 metric-related issues that could arise
> when a
> >>>>>>>>>>> Task is
> >>>>>>>>>>>>>>>>>>>> cancelled very early. (like, right away)
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> FLINK-6183 fixes a memory leak where the
> TaskMetricGroup
> >>>>>>> was
> >>>>>>>>>>>>>>> never closed
> >>>>>>>>>>>>>>>>>>>> FLINK-6184 fixes a NullPointerExceptions in the buffer
> >>>>>>>>>>> metrics
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> PR here: https://github.com/apache/flink/pull/3611
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On 26.03.2017 12:35, Aljoscha Krettek wrote:
> >>>>>>>>>>>>>>>>>>>>> I opened a PR for FLINK-6188:
> >>> https://github.com/apache/
> >>>>>>>>>>>>>>>>>>> flink/pull/3616
> >>>>>>>>>>>>>>>>>>>> <https://github.com/apache/flink/pull/3616>
> >>>>>>>>>>>>>>>>>>>>> This improves the previously very sparse test
> coverage
> >>> for
> >>>>>>>>>>>>>>>>>>>> timestamp/watermark assigners and fixes the bug.
> >>>>>>>>>>>>>>>>>>>>>> On 25 Mar 2017, at 10:22, Ufuk Celebi <
> uce@apache.org>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> I agree with Aljoscha.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> -1 because of FLINK-6188
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> On Sat, Mar 25, 2017 at 9:38 AM, Aljoscha Krettek <
> >>>>>>>>>>>>>>>>>>> aljoscha@apache.org>
> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>> I filed this issue, which was observed by a user:
> >>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6188
> >>>>>>>>>>>>>>>>>>>>>>> I think that’s blocking for 1.2.1.
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> On 24 Mar 2017, at 18:57, Ufuk Celebi <
> >>> uce@apache.org>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> RC1 doesn't contain Stefan's backport for the
> >>>>>>>>>>> Asynchronous
> >>>>>>>>>>>>>>> snapshots
> >>>>>>>>>>>>>>>>>>>>>>>> for heap-based keyed state that has been merged.
> >>> Should
> >>>>>>>>>>> we
> >>>>>>>>>>>>>>> create
> >>>>>>>>>>>>>>>>>>> RC2
> >>>>>>>>>>>>>>>>>>>>>>>> with that fix since the voting period only starts
> on
> >>>>>>>>>>> Monday?
> >>>>>>>>>>>>>>> I think
> >>>>>>>>>>>>>>>>>>>>>>>> it would only mean rerunning the scripts on your
> >>> side,
> >>>>>>>>>>>>> right?
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> – Ufuk
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> On Fri, Mar 24, 2017 at 3:05 PM, Robert Metzger <
> >>>>>>>>>>>>>>>>>>> rmetzger@apache.org>
> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>>>>> Dear Flink community,
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Please vote on releasing the following candidate
> as
> >>>>>>>>>>> Apache
> >>>>>>>>>>>>>>> Flink
> >>>>>>>>>>>>>>>>>>>> version 1.2
> >>>>>>>>>>>>>>>>>>>>>>>>> .1.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> The commit to be voted on:
> >>>>>>>>>>>>>>>>>>>>>>>>> *732e55bd* (*
> >>>>>>>>>>>>>>>>>>>> http://git-wip-us.apache.org/repos/asf/flink/commit/
> >>>>>>>>>>> 732e55bd
> >>>>>>>>>>>>>>>>>>>>>>>>> <http://git-wip-us.apache.org/
> >>>>>>>>>>>>> repos/asf/flink/commit/732e55b
> >>>>>>>>>>>>>>> d>*)
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> Branch:
> >>>>>>>>>>>>>>>>>>>>>>>>> release-1.2.1-rc1
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> The release artifacts to be voted on can be found
> >>> at:
> >>>>>>>>>>>>>>>>>>>>>>>>> *http://people.apache.org/~
> >>> rmetzger/flink-1.2.1-rc1/
> >>>>>>>>>>>>>>>>>>>>>>>>> <http://people.apache.org/~
> >>> rmetzger/flink-1.2.1-rc1/
> >>>>>>>> *
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> The release artifacts are signed with the key
> with
> >>>>>>>>>>>>>>> fingerprint
> >>>>>>>>>>>>>>>>>>>> D9839159:
> >>>>>>>>>>>>>>>>>>>>>>>>> http://www.apache.org/dist/flink/KEYS
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> The staging repository for this release can be
> found
> >>>>>>>>>>> at:
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> https://repository.apache.org/
> >>>>>>>>>>> content/repositories/orgapache
> >>>>>>>>>>>>>>> flink-1116
> >>>>>>>>>>>>>>>>>>>>>>>>> ------------------------------
> >>>>>>>>>>>>> ------------------------------
> >>>>>>>>>>>>>>> -
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> The vote ends on Wednesday, March 29, 2017, 3pm
> CET.
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>>> [ ] +1 Release this package as Apache Flink 1.2.1
> >>>>>>>>>>>>>>>>>>>>>>>>> [ ] -1 Do not release this package, because ...
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>
> >>>>>
> >>>
> >>>
> >
>
>
Re: [VOTE] Release Apache Flink 1.2.1 (RC1)
Posted by Aljoscha Krettek <al...@apache.org>.
The commits around FLINK-5808 have been reverted on release-1.2.
> On 4. Apr 2017, at 12:16, Stefan Richter <s....@data-artisans.com> wrote:
>
> I have created a custom build of RocksDB 4.11.2 that fixes a significant performance problem with append operations. I think this should definitely be part of the 1.2.1 release because this is already blocking some users. What is missing is uploading the jar to maven central and a testing run, e.g. with some misbehaved job that has large state.
>
>
>> Am 04.04.2017 um 11:57 schrieb Robert Metzger <rm...@apache.org>:
>>
>> Thank you for opening a PR for this.
>>
>> Chesnay, do you need more reviews for the metrics changes / backports?
>>
>> Are there any other release blockers for 1.2.1, or are we good to go?
>>
>> On Mon, Apr 3, 2017 at 6:48 PM, Aljoscha Krettek <al...@apache.org>
>> wrote:
>>
>>> I created a PR for the revert: https://github.com/apache/flink/pull/3664
>>>
>>>> On 3. Apr 2017, at 18:32, Stephan Ewen <se...@apache.org> wrote:
>>>>
>>>> +1 for options (1), but also invest the time to fix it properly for 1.2.2
>>>>
>>>>
>>>> On Mon, Apr 3, 2017 at 9:10 AM, Kostas Kloudas <
>>> k.kloudas@data-artisans.com>
>>>> wrote:
>>>>
>>>>> +1 for 1
>>>>>
>>>>>> On Apr 3, 2017, at 5:52 PM, Till Rohrmann <tr...@apache.org>
>>> wrote:
>>>>>>
>>>>>> +1 for option 1)
>>>>>>
>>>>>> On Mon, Apr 3, 2017 at 5:48 PM, Fabian Hueske <fh...@gmail.com>
>>> wrote:
>>>>>>
>>>>>>> +1 to option 1)
>>>>>>>
>>>>>>> 2017-04-03 16:57 GMT+02:00 Ted Yu <yu...@gmail.com>:
>>>>>>>
>>>>>>>> Looks like #1 is better - 1.2.1 would be at least as stable as 1.2.0
>>>>>>>>
>>>>>>>> Cheers
>>>>>>>>
>>>>>>>> On Mon, Apr 3, 2017 at 7:39 AM, Aljoscha Krettek <
>>> aljoscha@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Just so we’re all on the same page. ;-)
>>>>>>>>>
>>>>>>>>> There was https://issues.apache.org/jira/browse/FLINK-5808 which
>>> was
>>>>> a
>>>>>>>>> bug that we initially discovered in Flink 1.2 which was/is about
>>>>>>> missing
>>>>>>>>> verification for the correctness of the combination of parallelism
>>> and
>>>>>>>>> max-parallelism. Due to lacking test coverage this introduced two
>>> more
>>>>>>>> bugs:
>>>>>>>>> - https://issues.apache.org/jira/browse/FLINK-6188: Some
>>>>>>>>> setParallelism() methods can't cope with default parallelism
>>>>>>>>> - https://issues.apache.org/jira/browse/FLINK-6209:
>>>>>>>>> StreamPlanEnvironment always has a parallelism of 1
>>>>>>>>>
>>>>>>>>> IMHO, the options are:
>>>>>>>>> 1) revert the changes made for FLINK-5808 on the release-1.2 branch
>>>>>>> and
>>>>>>>>> live with the bug still being present
>>>>>>>>> 2) put in more work to fix FLINK-5808 which requires fixing some
>>>>>>>> problems
>>>>>>>>> that have existed for a long time with how the parallelism is set in
>>>>>>>>> streaming programs
>>>>>>>>>
>>>>>>>>> Best,
>>>>>>>>> Aljoscha
>>>>>>>>>
>>>>>>>>>> On 31. Mar 2017, at 21:34, Robert Metzger <rm...@apache.org>
>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> I don't know what is best to do, but I think releasing 1.2.1 with
>>>>>>>>>> potentially more bugs than 1.2.0 is not a good option.
>>>>>>>>>> I suspect a good workaround for FLINK-6188
>>>>>>>>>> <https://issues.apache.org/jira/browse/FLINK-6188> is setting the
>>>>>>>>>> parallelism manually for operators that can't cope with the default
>>>>>>> -1
>>>>>>>>>> parallelism.
>>>>>>>>>>
>>>>>>>>>> On Fri, Mar 31, 2017 at 9:06 PM, Aljoscha Krettek <
>>>>>>> aljoscha@apache.org
>>>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> You mean reverting the changes around FLINK-5808 [1]? This is what
>>>>>>>>>>> introduced the follow-up FLINK-6188 [2].
>>>>>>>>>>>
>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/FLINK-5808
>>>>>>>>>>> [2]https://issues.apache.org/jira/browse/FLINK-6188
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Mar 31, 2017, at 19:10, Robert Metzger wrote:
>>>>>>>>>>>> I think reverting FLINK-6188 for the 1.2 branch might be a good
>>>>>>> idea.
>>>>>>>>>>>> FLINK-6188 introduced two new bugs, so undoing the FLINK-6188 fix
>>>>>>>> will
>>>>>>>>>>>> lead
>>>>>>>>>>>> only to one known bug in 1.2.1, instead of an uncertain number of
>>>>>>>>> issues.
>>>>>>>>>>>> So 1.2.1 is not going to be worse than 1.2.0
>>>>>>>>>>>>
>>>>>>>>>>>> The fix will hopefully make it into 1.2.2 then.
>>>>>>>>>>>>
>>>>>>>>>>>> Any other thoughts on this?
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Mar 31, 2017 at 6:46 PM, Fabian Hueske <
>>> fhueske@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> I merged the fix for FLINK-6044 to the release-1.2 and
>>> release-1.1
>>>>>>>>>>> branch.
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2017-03-31 15:02 GMT+02:00 Fabian Hueske <fh...@gmail.com>:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> We should also backport the fix for FLINK-6044 to Flink 1.2.1.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I'll take care of that.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2017-03-30 18:50 GMT+02:00 Aljoscha Krettek <
>>> aljoscha@apache.org
>>>>>>>> :
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6188 turns out to
>>>>>>> be
>>>>>>>> a
>>>>>>>>>>> bit
>>>>>>>>>>>>>>> more involved, see my comments on the PR:
>>>>>>>>>>>>>>> https://github.com/apache/flink/pull/3616.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> As I said there, maybe we should revert the commits regarding
>>>>>>>>>>>>>>> parallelism/max-parallelism changes and release and then fix
>>> it
>>>>>>>>>>> later.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Mar 29, 2017, at 23:08, Aljoscha Krettek wrote:
>>>>>>>>>>>>>>>> I commented on FLINK-6214: I think it's working as intended,
>>>>>>>>>>> although
>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>> could fix the javadoc/doc.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Wed, Mar 29, 2017, at 17:35, Timo Walther wrote:
>>>>>>>>>>>>>>>>> A user reported that all tumbling and slinding window
>>>>>>> assigners
>>>>>>>>>>>>>>> contain
>>>>>>>>>>>>>>>>> a pretty obvious bug about offsets.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6214
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I think we should also fix this for 1.2.1. What do you
>>> think?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>> Timo
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Am 29/03/17 um 11:30 schrieb Robert Metzger:
>>>>>>>>>>>>>>>>>> Hi Haohui,
>>>>>>>>>>>>>>>>>> I agree that we should fix the parallelism issue.
>>> Otherwise,
>>>>>>>>>>> the
>>>>>>>>>>>>>>> 1.2.1
>>>>>>>>>>>>>>>>>> release would introduce a new bug.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Tue, Mar 28, 2017 at 11:59 PM, Haohui Mai <
>>>>>>>>>>> ricetons@gmail.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> -1 (non-binding)
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> We recently found out that all jobs submitted via UI will
>>>>>>>>>>> have a
>>>>>>>>>>>>>>>>>>> parallelism of 1, potentially due to FLINK-5808.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Filed FLINK-6209 to track it.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> ~Haohui
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Mon, Mar 27, 2017 at 2:59 AM Chesnay Schepler <
>>>>>>>>>>>>>>> chesnay@apache.org>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> If possible I would like to include FLINK-6183 &
>>> FLINK-6184
>>>>>>>>>>> as
>>>>>>>>>>>>>>> well.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> They fix 2 metric-related issues that could arise when a
>>>>>>>>>>> Task is
>>>>>>>>>>>>>>>>>>>> cancelled very early. (like, right away)
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> FLINK-6183 fixes a memory leak where the TaskMetricGroup
>>>>>>> was
>>>>>>>>>>>>>>> never closed
>>>>>>>>>>>>>>>>>>>> FLINK-6184 fixes a NullPointerExceptions in the buffer
>>>>>>>>>>> metrics
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> PR here: https://github.com/apache/flink/pull/3611
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On 26.03.2017 12:35, Aljoscha Krettek wrote:
>>>>>>>>>>>>>>>>>>>>> I opened a PR for FLINK-6188:
>>> https://github.com/apache/
>>>>>>>>>>>>>>>>>>> flink/pull/3616
>>>>>>>>>>>>>>>>>>>> <https://github.com/apache/flink/pull/3616>
>>>>>>>>>>>>>>>>>>>>> This improves the previously very sparse test coverage
>>> for
>>>>>>>>>>>>>>>>>>>> timestamp/watermark assigners and fixes the bug.
>>>>>>>>>>>>>>>>>>>>>> On 25 Mar 2017, at 10:22, Ufuk Celebi <uc...@apache.org>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I agree with Aljoscha.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> -1 because of FLINK-6188
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Sat, Mar 25, 2017 at 9:38 AM, Aljoscha Krettek <
>>>>>>>>>>>>>>>>>>> aljoscha@apache.org>
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>> I filed this issue, which was observed by a user:
>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6188
>>>>>>>>>>>>>>>>>>>>>>> I think that’s blocking for 1.2.1.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On 24 Mar 2017, at 18:57, Ufuk Celebi <
>>> uce@apache.org>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> RC1 doesn't contain Stefan's backport for the
>>>>>>>>>>> Asynchronous
>>>>>>>>>>>>>>> snapshots
>>>>>>>>>>>>>>>>>>>>>>>> for heap-based keyed state that has been merged.
>>> Should
>>>>>>>>>>> we
>>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>>>> RC2
>>>>>>>>>>>>>>>>>>>>>>>> with that fix since the voting period only starts on
>>>>>>>>>>> Monday?
>>>>>>>>>>>>>>> I think
>>>>>>>>>>>>>>>>>>>>>>>> it would only mean rerunning the scripts on your
>>> side,
>>>>>>>>>>>>> right?
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> – Ufuk
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Mar 24, 2017 at 3:05 PM, Robert Metzger <
>>>>>>>>>>>>>>>>>>> rmetzger@apache.org>
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>> Dear Flink community,
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Please vote on releasing the following candidate as
>>>>>>>>>>> Apache
>>>>>>>>>>>>>>> Flink
>>>>>>>>>>>>>>>>>>>> version 1.2
>>>>>>>>>>>>>>>>>>>>>>>>> .1.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> The commit to be voted on:
>>>>>>>>>>>>>>>>>>>>>>>>> *732e55bd* (*
>>>>>>>>>>>>>>>>>>>> http://git-wip-us.apache.org/repos/asf/flink/commit/
>>>>>>>>>>> 732e55bd
>>>>>>>>>>>>>>>>>>>>>>>>> <http://git-wip-us.apache.org/
>>>>>>>>>>>>> repos/asf/flink/commit/732e55b
>>>>>>>>>>>>>>> d>*)
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Branch:
>>>>>>>>>>>>>>>>>>>>>>>>> release-1.2.1-rc1
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> The release artifacts to be voted on can be found
>>> at:
>>>>>>>>>>>>>>>>>>>>>>>>> *http://people.apache.org/~
>>> rmetzger/flink-1.2.1-rc1/
>>>>>>>>>>>>>>>>>>>>>>>>> <http://people.apache.org/~
>>> rmetzger/flink-1.2.1-rc1/
>>>>>>>> *
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> The release artifacts are signed with the key with
>>>>>>>>>>>>>>> fingerprint
>>>>>>>>>>>>>>>>>>>> D9839159:
>>>>>>>>>>>>>>>>>>>>>>>>> http://www.apache.org/dist/flink/KEYS
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> The staging repository for this release can be found
>>>>>>>>>>> at:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> https://repository.apache.org/
>>>>>>>>>>> content/repositories/orgapache
>>>>>>>>>>>>>>> flink-1116
>>>>>>>>>>>>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>>> -
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> The vote ends on Wednesday, March 29, 2017, 3pm CET.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> [ ] +1 Release this package as Apache Flink 1.2.1
>>>>>>>>>>>>>>>>>>>>>>>>> [ ] -1 Do not release this package, because ...
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>
>>>
>
Re: [VOTE] Release Apache Flink 1.2.1 (RC1)
Posted by Stefan Richter <s....@data-artisans.com>.
I have created a custom build of RocksDB 4.11.2 that fixes a significant performance problem with append operations. I think this should definitely be part of the 1.2.1 release because this is already blocking some users. What is missing is uploading the jar to maven central and a testing run, e.g. with some misbehaved job that has large state.
> Am 04.04.2017 um 11:57 schrieb Robert Metzger <rm...@apache.org>:
>
> Thank you for opening a PR for this.
>
> Chesnay, do you need more reviews for the metrics changes / backports?
>
> Are there any other release blockers for 1.2.1, or are we good to go?
>
> On Mon, Apr 3, 2017 at 6:48 PM, Aljoscha Krettek <al...@apache.org>
> wrote:
>
>> I created a PR for the revert: https://github.com/apache/flink/pull/3664
>>
>>> On 3. Apr 2017, at 18:32, Stephan Ewen <se...@apache.org> wrote:
>>>
>>> +1 for options (1), but also invest the time to fix it properly for 1.2.2
>>>
>>>
>>> On Mon, Apr 3, 2017 at 9:10 AM, Kostas Kloudas <
>> k.kloudas@data-artisans.com>
>>> wrote:
>>>
>>>> +1 for 1
>>>>
>>>>> On Apr 3, 2017, at 5:52 PM, Till Rohrmann <tr...@apache.org>
>> wrote:
>>>>>
>>>>> +1 for option 1)
>>>>>
>>>>> On Mon, Apr 3, 2017 at 5:48 PM, Fabian Hueske <fh...@gmail.com>
>> wrote:
>>>>>
>>>>>> +1 to option 1)
>>>>>>
>>>>>> 2017-04-03 16:57 GMT+02:00 Ted Yu <yu...@gmail.com>:
>>>>>>
>>>>>>> Looks like #1 is better - 1.2.1 would be at least as stable as 1.2.0
>>>>>>>
>>>>>>> Cheers
>>>>>>>
>>>>>>> On Mon, Apr 3, 2017 at 7:39 AM, Aljoscha Krettek <
>> aljoscha@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Just so we’re all on the same page. ;-)
>>>>>>>>
>>>>>>>> There was https://issues.apache.org/jira/browse/FLINK-5808 which
>> was
>>>> a
>>>>>>>> bug that we initially discovered in Flink 1.2 which was/is about
>>>>>> missing
>>>>>>>> verification for the correctness of the combination of parallelism
>> and
>>>>>>>> max-parallelism. Due to lacking test coverage this introduced two
>> more
>>>>>>> bugs:
>>>>>>>> - https://issues.apache.org/jira/browse/FLINK-6188: Some
>>>>>>>> setParallelism() methods can't cope with default parallelism
>>>>>>>> - https://issues.apache.org/jira/browse/FLINK-6209:
>>>>>>>> StreamPlanEnvironment always has a parallelism of 1
>>>>>>>>
>>>>>>>> IMHO, the options are:
>>>>>>>> 1) revert the changes made for FLINK-5808 on the release-1.2 branch
>>>>>> and
>>>>>>>> live with the bug still being present
>>>>>>>> 2) put in more work to fix FLINK-5808 which requires fixing some
>>>>>>> problems
>>>>>>>> that have existed for a long time with how the parallelism is set in
>>>>>>>> streaming programs
>>>>>>>>
>>>>>>>> Best,
>>>>>>>> Aljoscha
>>>>>>>>
>>>>>>>>> On 31. Mar 2017, at 21:34, Robert Metzger <rm...@apache.org>
>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> I don't know what is best to do, but I think releasing 1.2.1 with
>>>>>>>>> potentially more bugs than 1.2.0 is not a good option.
>>>>>>>>> I suspect a good workaround for FLINK-6188
>>>>>>>>> <https://issues.apache.org/jira/browse/FLINK-6188> is setting the
>>>>>>>>> parallelism manually for operators that can't cope with the default
>>>>>> -1
>>>>>>>>> parallelism.
>>>>>>>>>
>>>>>>>>> On Fri, Mar 31, 2017 at 9:06 PM, Aljoscha Krettek <
>>>>>> aljoscha@apache.org
>>>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> You mean reverting the changes around FLINK-5808 [1]? This is what
>>>>>>>>>> introduced the follow-up FLINK-6188 [2].
>>>>>>>>>>
>>>>>>>>>> [1] https://issues.apache.org/jira/browse/FLINK-5808
>>>>>>>>>> [2]https://issues.apache.org/jira/browse/FLINK-6188
>>>>>>>>>>
>>>>>>>>>> On Fri, Mar 31, 2017, at 19:10, Robert Metzger wrote:
>>>>>>>>>>> I think reverting FLINK-6188 for the 1.2 branch might be a good
>>>>>> idea.
>>>>>>>>>>> FLINK-6188 introduced two new bugs, so undoing the FLINK-6188 fix
>>>>>>> will
>>>>>>>>>>> lead
>>>>>>>>>>> only to one known bug in 1.2.1, instead of an uncertain number of
>>>>>>>> issues.
>>>>>>>>>>> So 1.2.1 is not going to be worse than 1.2.0
>>>>>>>>>>>
>>>>>>>>>>> The fix will hopefully make it into 1.2.2 then.
>>>>>>>>>>>
>>>>>>>>>>> Any other thoughts on this?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Mar 31, 2017 at 6:46 PM, Fabian Hueske <
>> fhueske@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I merged the fix for FLINK-6044 to the release-1.2 and
>> release-1.1
>>>>>>>>>> branch.
>>>>>>>>>>>>
>>>>>>>>>>>> 2017-03-31 15:02 GMT+02:00 Fabian Hueske <fh...@gmail.com>:
>>>>>>>>>>>>
>>>>>>>>>>>>> We should also backport the fix for FLINK-6044 to Flink 1.2.1.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I'll take care of that.
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2017-03-30 18:50 GMT+02:00 Aljoscha Krettek <
>> aljoscha@apache.org
>>>>>>> :
>>>>>>>>>>>>>
>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6188 turns out to
>>>>>> be
>>>>>>> a
>>>>>>>>>> bit
>>>>>>>>>>>>>> more involved, see my comments on the PR:
>>>>>>>>>>>>>> https://github.com/apache/flink/pull/3616.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> As I said there, maybe we should revert the commits regarding
>>>>>>>>>>>>>> parallelism/max-parallelism changes and release and then fix
>> it
>>>>>>>>>> later.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Mar 29, 2017, at 23:08, Aljoscha Krettek wrote:
>>>>>>>>>>>>>>> I commented on FLINK-6214: I think it's working as intended,
>>>>>>>>>> although
>>>>>>>>>>>> we
>>>>>>>>>>>>>>> could fix the javadoc/doc.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Mar 29, 2017, at 17:35, Timo Walther wrote:
>>>>>>>>>>>>>>>> A user reported that all tumbling and slinding window
>>>>>> assigners
>>>>>>>>>>>>>> contain
>>>>>>>>>>>>>>>> a pretty obvious bug about offsets.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6214
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I think we should also fix this for 1.2.1. What do you
>> think?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>> Timo
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Am 29/03/17 um 11:30 schrieb Robert Metzger:
>>>>>>>>>>>>>>>>> Hi Haohui,
>>>>>>>>>>>>>>>>> I agree that we should fix the parallelism issue.
>> Otherwise,
>>>>>>>>>> the
>>>>>>>>>>>>>> 1.2.1
>>>>>>>>>>>>>>>>> release would introduce a new bug.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Tue, Mar 28, 2017 at 11:59 PM, Haohui Mai <
>>>>>>>>>> ricetons@gmail.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> -1 (non-binding)
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> We recently found out that all jobs submitted via UI will
>>>>>>>>>> have a
>>>>>>>>>>>>>>>>>> parallelism of 1, potentially due to FLINK-5808.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Filed FLINK-6209 to track it.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> ~Haohui
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Mon, Mar 27, 2017 at 2:59 AM Chesnay Schepler <
>>>>>>>>>>>>>> chesnay@apache.org>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> If possible I would like to include FLINK-6183 &
>> FLINK-6184
>>>>>>>>>> as
>>>>>>>>>>>>>> well.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> They fix 2 metric-related issues that could arise when a
>>>>>>>>>> Task is
>>>>>>>>>>>>>>>>>>> cancelled very early. (like, right away)
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> FLINK-6183 fixes a memory leak where the TaskMetricGroup
>>>>>> was
>>>>>>>>>>>>>> never closed
>>>>>>>>>>>>>>>>>>> FLINK-6184 fixes a NullPointerExceptions in the buffer
>>>>>>>>>> metrics
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> PR here: https://github.com/apache/flink/pull/3611
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On 26.03.2017 12:35, Aljoscha Krettek wrote:
>>>>>>>>>>>>>>>>>>>> I opened a PR for FLINK-6188:
>> https://github.com/apache/
>>>>>>>>>>>>>>>>>> flink/pull/3616
>>>>>>>>>>>>>>>>>>> <https://github.com/apache/flink/pull/3616>
>>>>>>>>>>>>>>>>>>>> This improves the previously very sparse test coverage
>> for
>>>>>>>>>>>>>>>>>>> timestamp/watermark assigners and fixes the bug.
>>>>>>>>>>>>>>>>>>>>> On 25 Mar 2017, at 10:22, Ufuk Celebi <uc...@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I agree with Aljoscha.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> -1 because of FLINK-6188
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Sat, Mar 25, 2017 at 9:38 AM, Aljoscha Krettek <
>>>>>>>>>>>>>>>>>> aljoscha@apache.org>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>> I filed this issue, which was observed by a user:
>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6188
>>>>>>>>>>>>>>>>>>>>>> I think that’s blocking for 1.2.1.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On 24 Mar 2017, at 18:57, Ufuk Celebi <
>> uce@apache.org>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> RC1 doesn't contain Stefan's backport for the
>>>>>>>>>> Asynchronous
>>>>>>>>>>>>>> snapshots
>>>>>>>>>>>>>>>>>>>>>>> for heap-based keyed state that has been merged.
>> Should
>>>>>>>>>> we
>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>>>> RC2
>>>>>>>>>>>>>>>>>>>>>>> with that fix since the voting period only starts on
>>>>>>>>>> Monday?
>>>>>>>>>>>>>> I think
>>>>>>>>>>>>>>>>>>>>>>> it would only mean rerunning the scripts on your
>> side,
>>>>>>>>>>>> right?
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> – Ufuk
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On Fri, Mar 24, 2017 at 3:05 PM, Robert Metzger <
>>>>>>>>>>>>>>>>>> rmetzger@apache.org>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>>> Dear Flink community,
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Please vote on releasing the following candidate as
>>>>>>>>>> Apache
>>>>>>>>>>>>>> Flink
>>>>>>>>>>>>>>>>>>> version 1.2
>>>>>>>>>>>>>>>>>>>>>>>> .1.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> The commit to be voted on:
>>>>>>>>>>>>>>>>>>>>>>>> *732e55bd* (*
>>>>>>>>>>>>>>>>>>> http://git-wip-us.apache.org/repos/asf/flink/commit/
>>>>>>>>>> 732e55bd
>>>>>>>>>>>>>>>>>>>>>>>> <http://git-wip-us.apache.org/
>>>>>>>>>>>> repos/asf/flink/commit/732e55b
>>>>>>>>>>>>>> d>*)
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Branch:
>>>>>>>>>>>>>>>>>>>>>>>> release-1.2.1-rc1
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> The release artifacts to be voted on can be found
>> at:
>>>>>>>>>>>>>>>>>>>>>>>> *http://people.apache.org/~
>> rmetzger/flink-1.2.1-rc1/
>>>>>>>>>>>>>>>>>>>>>>>> <http://people.apache.org/~
>> rmetzger/flink-1.2.1-rc1/
>>>>>>> *
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> The release artifacts are signed with the key with
>>>>>>>>>>>>>> fingerprint
>>>>>>>>>>>>>>>>>>> D9839159:
>>>>>>>>>>>>>>>>>>>>>>>> http://www.apache.org/dist/flink/KEYS
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> The staging repository for this release can be found
>>>>>>>>>> at:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> https://repository.apache.org/
>>>>>>>>>> content/repositories/orgapache
>>>>>>>>>>>>>> flink-1116
>>>>>>>>>>>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>> -
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> The vote ends on Wednesday, March 29, 2017, 3pm CET.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> [ ] +1 Release this package as Apache Flink 1.2.1
>>>>>>>>>>>>>>>>>>>>>>>> [ ] -1 Do not release this package, because ...
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>>
>>
>>
Re: [VOTE] Release Apache Flink 1.2.1 (RC1)
Posted by Robert Metzger <rm...@apache.org>.
Thank you for opening a PR for this.
Chesnay, do you need more reviews for the metrics changes / backports?
Are there any other release blockers for 1.2.1, or are we good to go?
On Mon, Apr 3, 2017 at 6:48 PM, Aljoscha Krettek <al...@apache.org>
wrote:
> I created a PR for the revert: https://github.com/apache/flink/pull/3664
>
> > On 3. Apr 2017, at 18:32, Stephan Ewen <se...@apache.org> wrote:
> >
> > +1 for options (1), but also invest the time to fix it properly for 1.2.2
> >
> >
> > On Mon, Apr 3, 2017 at 9:10 AM, Kostas Kloudas <
> k.kloudas@data-artisans.com>
> > wrote:
> >
> >> +1 for 1
> >>
> >>> On Apr 3, 2017, at 5:52 PM, Till Rohrmann <tr...@apache.org>
> wrote:
> >>>
> >>> +1 for option 1)
> >>>
> >>> On Mon, Apr 3, 2017 at 5:48 PM, Fabian Hueske <fh...@gmail.com>
> wrote:
> >>>
> >>>> +1 to option 1)
> >>>>
> >>>> 2017-04-03 16:57 GMT+02:00 Ted Yu <yu...@gmail.com>:
> >>>>
> >>>>> Looks like #1 is better - 1.2.1 would be at least as stable as 1.2.0
> >>>>>
> >>>>> Cheers
> >>>>>
> >>>>> On Mon, Apr 3, 2017 at 7:39 AM, Aljoscha Krettek <
> aljoscha@apache.org>
> >>>>> wrote:
> >>>>>
> >>>>>> Just so we’re all on the same page. ;-)
> >>>>>>
> >>>>>> There was https://issues.apache.org/jira/browse/FLINK-5808 which
> was
> >> a
> >>>>>> bug that we initially discovered in Flink 1.2 which was/is about
> >>>> missing
> >>>>>> verification for the correctness of the combination of parallelism
> and
> >>>>>> max-parallelism. Due to lacking test coverage this introduced two
> more
> >>>>> bugs:
> >>>>>> - https://issues.apache.org/jira/browse/FLINK-6188: Some
> >>>>>> setParallelism() methods can't cope with default parallelism
> >>>>>> - https://issues.apache.org/jira/browse/FLINK-6209:
> >>>>>> StreamPlanEnvironment always has a parallelism of 1
> >>>>>>
> >>>>>> IMHO, the options are:
> >>>>>> 1) revert the changes made for FLINK-5808 on the release-1.2 branch
> >>>> and
> >>>>>> live with the bug still being present
> >>>>>> 2) put in more work to fix FLINK-5808 which requires fixing some
> >>>>> problems
> >>>>>> that have existed for a long time with how the parallelism is set in
> >>>>>> streaming programs
> >>>>>>
> >>>>>> Best,
> >>>>>> Aljoscha
> >>>>>>
> >>>>>>> On 31. Mar 2017, at 21:34, Robert Metzger <rm...@apache.org>
> >>>> wrote:
> >>>>>>>
> >>>>>>> I don't know what is best to do, but I think releasing 1.2.1 with
> >>>>>>> potentially more bugs than 1.2.0 is not a good option.
> >>>>>>> I suspect a good workaround for FLINK-6188
> >>>>>>> <https://issues.apache.org/jira/browse/FLINK-6188> is setting the
> >>>>>>> parallelism manually for operators that can't cope with the default
> >>>> -1
> >>>>>>> parallelism.
> >>>>>>>
> >>>>>>> On Fri, Mar 31, 2017 at 9:06 PM, Aljoscha Krettek <
> >>>> aljoscha@apache.org
> >>>>>>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> You mean reverting the changes around FLINK-5808 [1]? This is what
> >>>>>>>> introduced the follow-up FLINK-6188 [2].
> >>>>>>>>
> >>>>>>>> [1] https://issues.apache.org/jira/browse/FLINK-5808
> >>>>>>>> [2]https://issues.apache.org/jira/browse/FLINK-6188
> >>>>>>>>
> >>>>>>>> On Fri, Mar 31, 2017, at 19:10, Robert Metzger wrote:
> >>>>>>>>> I think reverting FLINK-6188 for the 1.2 branch might be a good
> >>>> idea.
> >>>>>>>>> FLINK-6188 introduced two new bugs, so undoing the FLINK-6188 fix
> >>>>> will
> >>>>>>>>> lead
> >>>>>>>>> only to one known bug in 1.2.1, instead of an uncertain number of
> >>>>>> issues.
> >>>>>>>>> So 1.2.1 is not going to be worse than 1.2.0
> >>>>>>>>>
> >>>>>>>>> The fix will hopefully make it into 1.2.2 then.
> >>>>>>>>>
> >>>>>>>>> Any other thoughts on this?
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Fri, Mar 31, 2017 at 6:46 PM, Fabian Hueske <
> fhueske@gmail.com>
> >>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> I merged the fix for FLINK-6044 to the release-1.2 and
> release-1.1
> >>>>>>>> branch.
> >>>>>>>>>>
> >>>>>>>>>> 2017-03-31 15:02 GMT+02:00 Fabian Hueske <fh...@gmail.com>:
> >>>>>>>>>>
> >>>>>>>>>>> We should also backport the fix for FLINK-6044 to Flink 1.2.1.
> >>>>>>>>>>>
> >>>>>>>>>>> I'll take care of that.
> >>>>>>>>>>>
> >>>>>>>>>>> 2017-03-30 18:50 GMT+02:00 Aljoscha Krettek <
> aljoscha@apache.org
> >>>>> :
> >>>>>>>>>>>
> >>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6188 turns out to
> >>>> be
> >>>>> a
> >>>>>>>> bit
> >>>>>>>>>>>> more involved, see my comments on the PR:
> >>>>>>>>>>>> https://github.com/apache/flink/pull/3616.
> >>>>>>>>>>>>
> >>>>>>>>>>>> As I said there, maybe we should revert the commits regarding
> >>>>>>>>>>>> parallelism/max-parallelism changes and release and then fix
> it
> >>>>>>>> later.
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Wed, Mar 29, 2017, at 23:08, Aljoscha Krettek wrote:
> >>>>>>>>>>>>> I commented on FLINK-6214: I think it's working as intended,
> >>>>>>>> although
> >>>>>>>>>> we
> >>>>>>>>>>>>> could fix the javadoc/doc.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Wed, Mar 29, 2017, at 17:35, Timo Walther wrote:
> >>>>>>>>>>>>>> A user reported that all tumbling and slinding window
> >>>> assigners
> >>>>>>>>>>>> contain
> >>>>>>>>>>>>>> a pretty obvious bug about offsets.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6214
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I think we should also fix this for 1.2.1. What do you
> think?
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Regards,
> >>>>>>>>>>>>>> Timo
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Am 29/03/17 um 11:30 schrieb Robert Metzger:
> >>>>>>>>>>>>>>> Hi Haohui,
> >>>>>>>>>>>>>>> I agree that we should fix the parallelism issue.
> Otherwise,
> >>>>>>>> the
> >>>>>>>>>>>> 1.2.1
> >>>>>>>>>>>>>>> release would introduce a new bug.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Tue, Mar 28, 2017 at 11:59 PM, Haohui Mai <
> >>>>>>>> ricetons@gmail.com>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> -1 (non-binding)
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> We recently found out that all jobs submitted via UI will
> >>>>>>>> have a
> >>>>>>>>>>>>>>>> parallelism of 1, potentially due to FLINK-5808.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Filed FLINK-6209 to track it.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> ~Haohui
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Mon, Mar 27, 2017 at 2:59 AM Chesnay Schepler <
> >>>>>>>>>>>> chesnay@apache.org>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> If possible I would like to include FLINK-6183 &
> FLINK-6184
> >>>>>>>> as
> >>>>>>>>>>>> well.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> They fix 2 metric-related issues that could arise when a
> >>>>>>>> Task is
> >>>>>>>>>>>>>>>>> cancelled very early. (like, right away)
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> FLINK-6183 fixes a memory leak where the TaskMetricGroup
> >>>> was
> >>>>>>>>>>>> never closed
> >>>>>>>>>>>>>>>>> FLINK-6184 fixes a NullPointerExceptions in the buffer
> >>>>>>>> metrics
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> PR here: https://github.com/apache/flink/pull/3611
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On 26.03.2017 12:35, Aljoscha Krettek wrote:
> >>>>>>>>>>>>>>>>>> I opened a PR for FLINK-6188:
> https://github.com/apache/
> >>>>>>>>>>>>>>>> flink/pull/3616
> >>>>>>>>>>>>>>>>> <https://github.com/apache/flink/pull/3616>
> >>>>>>>>>>>>>>>>>> This improves the previously very sparse test coverage
> for
> >>>>>>>>>>>>>>>>> timestamp/watermark assigners and fixes the bug.
> >>>>>>>>>>>>>>>>>>> On 25 Mar 2017, at 10:22, Ufuk Celebi <uc...@apache.org>
> >>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> I agree with Aljoscha.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> -1 because of FLINK-6188
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Sat, Mar 25, 2017 at 9:38 AM, Aljoscha Krettek <
> >>>>>>>>>>>>>>>> aljoscha@apache.org>
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>> I filed this issue, which was observed by a user:
> >>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6188
> >>>>>>>>>>>>>>>>>>>> I think that’s blocking for 1.2.1.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On 24 Mar 2017, at 18:57, Ufuk Celebi <
> uce@apache.org>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> RC1 doesn't contain Stefan's backport for the
> >>>>>>>> Asynchronous
> >>>>>>>>>>>> snapshots
> >>>>>>>>>>>>>>>>>>>>> for heap-based keyed state that has been merged.
> Should
> >>>>>>>> we
> >>>>>>>>>>>> create
> >>>>>>>>>>>>>>>> RC2
> >>>>>>>>>>>>>>>>>>>>> with that fix since the voting period only starts on
> >>>>>>>> Monday?
> >>>>>>>>>>>> I think
> >>>>>>>>>>>>>>>>>>>>> it would only mean rerunning the scripts on your
> side,
> >>>>>>>>>> right?
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> – Ufuk
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> On Fri, Mar 24, 2017 at 3:05 PM, Robert Metzger <
> >>>>>>>>>>>>>>>> rmetzger@apache.org>
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>>> Dear Flink community,
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Please vote on releasing the following candidate as
> >>>>>>>> Apache
> >>>>>>>>>>>> Flink
> >>>>>>>>>>>>>>>>> version 1.2
> >>>>>>>>>>>>>>>>>>>>>> .1.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> The commit to be voted on:
> >>>>>>>>>>>>>>>>>>>>>> *732e55bd* (*
> >>>>>>>>>>>>>>>>> http://git-wip-us.apache.org/repos/asf/flink/commit/
> >>>>>>>> 732e55bd
> >>>>>>>>>>>>>>>>>>>>>> <http://git-wip-us.apache.org/
> >>>>>>>>>> repos/asf/flink/commit/732e55b
> >>>>>>>>>>>> d>*)
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Branch:
> >>>>>>>>>>>>>>>>>>>>>> release-1.2.1-rc1
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> The release artifacts to be voted on can be found
> at:
> >>>>>>>>>>>>>>>>>>>>>> *http://people.apache.org/~
> rmetzger/flink-1.2.1-rc1/
> >>>>>>>>>>>>>>>>>>>>>> <http://people.apache.org/~
> rmetzger/flink-1.2.1-rc1/
> >>>>> *
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> The release artifacts are signed with the key with
> >>>>>>>>>>>> fingerprint
> >>>>>>>>>>>>>>>>> D9839159:
> >>>>>>>>>>>>>>>>>>>>>> http://www.apache.org/dist/flink/KEYS
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> The staging repository for this release can be found
> >>>>>>>> at:
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> https://repository.apache.org/
> >>>>>>>> content/repositories/orgapache
> >>>>>>>>>>>> flink-1116
> >>>>>>>>>>>>>>>>>>>>>> ------------------------------
> >>>>>>>>>> ------------------------------
> >>>>>>>>>>>> -
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> The vote ends on Wednesday, March 29, 2017, 3pm CET.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> [ ] +1 Release this package as Apache Flink 1.2.1
> >>>>>>>>>>>>>>>>>>>>>> [ ] -1 Do not release this package, because ...
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>
> >>
>
>
Re: [VOTE] Release Apache Flink 1.2.1 (RC1)
Posted by Aljoscha Krettek <al...@apache.org>.
I created a PR for the revert: https://github.com/apache/flink/pull/3664
> On 3. Apr 2017, at 18:32, Stephan Ewen <se...@apache.org> wrote:
>
> +1 for options (1), but also invest the time to fix it properly for 1.2.2
>
>
> On Mon, Apr 3, 2017 at 9:10 AM, Kostas Kloudas <k....@data-artisans.com>
> wrote:
>
>> +1 for 1
>>
>>> On Apr 3, 2017, at 5:52 PM, Till Rohrmann <tr...@apache.org> wrote:
>>>
>>> +1 for option 1)
>>>
>>> On Mon, Apr 3, 2017 at 5:48 PM, Fabian Hueske <fh...@gmail.com> wrote:
>>>
>>>> +1 to option 1)
>>>>
>>>> 2017-04-03 16:57 GMT+02:00 Ted Yu <yu...@gmail.com>:
>>>>
>>>>> Looks like #1 is better - 1.2.1 would be at least as stable as 1.2.0
>>>>>
>>>>> Cheers
>>>>>
>>>>> On Mon, Apr 3, 2017 at 7:39 AM, Aljoscha Krettek <al...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> Just so we’re all on the same page. ;-)
>>>>>>
>>>>>> There was https://issues.apache.org/jira/browse/FLINK-5808 which was
>> a
>>>>>> bug that we initially discovered in Flink 1.2 which was/is about
>>>> missing
>>>>>> verification for the correctness of the combination of parallelism and
>>>>>> max-parallelism. Due to lacking test coverage this introduced two more
>>>>> bugs:
>>>>>> - https://issues.apache.org/jira/browse/FLINK-6188: Some
>>>>>> setParallelism() methods can't cope with default parallelism
>>>>>> - https://issues.apache.org/jira/browse/FLINK-6209:
>>>>>> StreamPlanEnvironment always has a parallelism of 1
>>>>>>
>>>>>> IMHO, the options are:
>>>>>> 1) revert the changes made for FLINK-5808 on the release-1.2 branch
>>>> and
>>>>>> live with the bug still being present
>>>>>> 2) put in more work to fix FLINK-5808 which requires fixing some
>>>>> problems
>>>>>> that have existed for a long time with how the parallelism is set in
>>>>>> streaming programs
>>>>>>
>>>>>> Best,
>>>>>> Aljoscha
>>>>>>
>>>>>>> On 31. Mar 2017, at 21:34, Robert Metzger <rm...@apache.org>
>>>> wrote:
>>>>>>>
>>>>>>> I don't know what is best to do, but I think releasing 1.2.1 with
>>>>>>> potentially more bugs than 1.2.0 is not a good option.
>>>>>>> I suspect a good workaround for FLINK-6188
>>>>>>> <https://issues.apache.org/jira/browse/FLINK-6188> is setting the
>>>>>>> parallelism manually for operators that can't cope with the default
>>>> -1
>>>>>>> parallelism.
>>>>>>>
>>>>>>> On Fri, Mar 31, 2017 at 9:06 PM, Aljoscha Krettek <
>>>> aljoscha@apache.org
>>>>>>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> You mean reverting the changes around FLINK-5808 [1]? This is what
>>>>>>>> introduced the follow-up FLINK-6188 [2].
>>>>>>>>
>>>>>>>> [1] https://issues.apache.org/jira/browse/FLINK-5808
>>>>>>>> [2]https://issues.apache.org/jira/browse/FLINK-6188
>>>>>>>>
>>>>>>>> On Fri, Mar 31, 2017, at 19:10, Robert Metzger wrote:
>>>>>>>>> I think reverting FLINK-6188 for the 1.2 branch might be a good
>>>> idea.
>>>>>>>>> FLINK-6188 introduced two new bugs, so undoing the FLINK-6188 fix
>>>>> will
>>>>>>>>> lead
>>>>>>>>> only to one known bug in 1.2.1, instead of an uncertain number of
>>>>>> issues.
>>>>>>>>> So 1.2.1 is not going to be worse than 1.2.0
>>>>>>>>>
>>>>>>>>> The fix will hopefully make it into 1.2.2 then.
>>>>>>>>>
>>>>>>>>> Any other thoughts on this?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Fri, Mar 31, 2017 at 6:46 PM, Fabian Hueske <fh...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> I merged the fix for FLINK-6044 to the release-1.2 and release-1.1
>>>>>>>> branch.
>>>>>>>>>>
>>>>>>>>>> 2017-03-31 15:02 GMT+02:00 Fabian Hueske <fh...@gmail.com>:
>>>>>>>>>>
>>>>>>>>>>> We should also backport the fix for FLINK-6044 to Flink 1.2.1.
>>>>>>>>>>>
>>>>>>>>>>> I'll take care of that.
>>>>>>>>>>>
>>>>>>>>>>> 2017-03-30 18:50 GMT+02:00 Aljoscha Krettek <aljoscha@apache.org
>>>>> :
>>>>>>>>>>>
>>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6188 turns out to
>>>> be
>>>>> a
>>>>>>>> bit
>>>>>>>>>>>> more involved, see my comments on the PR:
>>>>>>>>>>>> https://github.com/apache/flink/pull/3616.
>>>>>>>>>>>>
>>>>>>>>>>>> As I said there, maybe we should revert the commits regarding
>>>>>>>>>>>> parallelism/max-parallelism changes and release and then fix it
>>>>>>>> later.
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Mar 29, 2017, at 23:08, Aljoscha Krettek wrote:
>>>>>>>>>>>>> I commented on FLINK-6214: I think it's working as intended,
>>>>>>>> although
>>>>>>>>>> we
>>>>>>>>>>>>> could fix the javadoc/doc.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Mar 29, 2017, at 17:35, Timo Walther wrote:
>>>>>>>>>>>>>> A user reported that all tumbling and slinding window
>>>> assigners
>>>>>>>>>>>> contain
>>>>>>>>>>>>>> a pretty obvious bug about offsets.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6214
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I think we should also fix this for 1.2.1. What do you think?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>> Timo
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Am 29/03/17 um 11:30 schrieb Robert Metzger:
>>>>>>>>>>>>>>> Hi Haohui,
>>>>>>>>>>>>>>> I agree that we should fix the parallelism issue. Otherwise,
>>>>>>>> the
>>>>>>>>>>>> 1.2.1
>>>>>>>>>>>>>>> release would introduce a new bug.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Tue, Mar 28, 2017 at 11:59 PM, Haohui Mai <
>>>>>>>> ricetons@gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> -1 (non-binding)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> We recently found out that all jobs submitted via UI will
>>>>>>>> have a
>>>>>>>>>>>>>>>> parallelism of 1, potentially due to FLINK-5808.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Filed FLINK-6209 to track it.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> ~Haohui
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Mon, Mar 27, 2017 at 2:59 AM Chesnay Schepler <
>>>>>>>>>>>> chesnay@apache.org>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> If possible I would like to include FLINK-6183 & FLINK-6184
>>>>>>>> as
>>>>>>>>>>>> well.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> They fix 2 metric-related issues that could arise when a
>>>>>>>> Task is
>>>>>>>>>>>>>>>>> cancelled very early. (like, right away)
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> FLINK-6183 fixes a memory leak where the TaskMetricGroup
>>>> was
>>>>>>>>>>>> never closed
>>>>>>>>>>>>>>>>> FLINK-6184 fixes a NullPointerExceptions in the buffer
>>>>>>>> metrics
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> PR here: https://github.com/apache/flink/pull/3611
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On 26.03.2017 12:35, Aljoscha Krettek wrote:
>>>>>>>>>>>>>>>>>> I opened a PR for FLINK-6188: https://github.com/apache/
>>>>>>>>>>>>>>>> flink/pull/3616
>>>>>>>>>>>>>>>>> <https://github.com/apache/flink/pull/3616>
>>>>>>>>>>>>>>>>>> This improves the previously very sparse test coverage for
>>>>>>>>>>>>>>>>> timestamp/watermark assigners and fixes the bug.
>>>>>>>>>>>>>>>>>>> On 25 Mar 2017, at 10:22, Ufuk Celebi <uc...@apache.org>
>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I agree with Aljoscha.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> -1 because of FLINK-6188
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Sat, Mar 25, 2017 at 9:38 AM, Aljoscha Krettek <
>>>>>>>>>>>>>>>> aljoscha@apache.org>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>> I filed this issue, which was observed by a user:
>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6188
>>>>>>>>>>>>>>>>>>>> I think that’s blocking for 1.2.1.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On 24 Mar 2017, at 18:57, Ufuk Celebi <uc...@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> RC1 doesn't contain Stefan's backport for the
>>>>>>>> Asynchronous
>>>>>>>>>>>> snapshots
>>>>>>>>>>>>>>>>>>>>> for heap-based keyed state that has been merged. Should
>>>>>>>> we
>>>>>>>>>>>> create
>>>>>>>>>>>>>>>> RC2
>>>>>>>>>>>>>>>>>>>>> with that fix since the voting period only starts on
>>>>>>>> Monday?
>>>>>>>>>>>> I think
>>>>>>>>>>>>>>>>>>>>> it would only mean rerunning the scripts on your side,
>>>>>>>>>> right?
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> – Ufuk
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Fri, Mar 24, 2017 at 3:05 PM, Robert Metzger <
>>>>>>>>>>>>>>>> rmetzger@apache.org>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>> Dear Flink community,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Please vote on releasing the following candidate as
>>>>>>>> Apache
>>>>>>>>>>>> Flink
>>>>>>>>>>>>>>>>> version 1.2
>>>>>>>>>>>>>>>>>>>>>> .1.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> The commit to be voted on:
>>>>>>>>>>>>>>>>>>>>>> *732e55bd* (*
>>>>>>>>>>>>>>>>> http://git-wip-us.apache.org/repos/asf/flink/commit/
>>>>>>>> 732e55bd
>>>>>>>>>>>>>>>>>>>>>> <http://git-wip-us.apache.org/
>>>>>>>>>> repos/asf/flink/commit/732e55b
>>>>>>>>>>>> d>*)
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Branch:
>>>>>>>>>>>>>>>>>>>>>> release-1.2.1-rc1
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> The release artifacts to be voted on can be found at:
>>>>>>>>>>>>>>>>>>>>>> *http://people.apache.org/~rmetzger/flink-1.2.1-rc1/
>>>>>>>>>>>>>>>>>>>>>> <http://people.apache.org/~rmetzger/flink-1.2.1-rc1/
>>>>> *
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> The release artifacts are signed with the key with
>>>>>>>>>>>> fingerprint
>>>>>>>>>>>>>>>>> D9839159:
>>>>>>>>>>>>>>>>>>>>>> http://www.apache.org/dist/flink/KEYS
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> The staging repository for this release can be found
>>>>>>>> at:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> https://repository.apache.org/
>>>>>>>> content/repositories/orgapache
>>>>>>>>>>>> flink-1116
>>>>>>>>>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>> ------------------------------
>>>>>>>>>>>> -
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> The vote ends on Wednesday, March 29, 2017, 3pm CET.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> [ ] +1 Release this package as Apache Flink 1.2.1
>>>>>>>>>>>>>>>>>>>>>> [ ] -1 Do not release this package, because ...
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>
>>
Re: [VOTE] Release Apache Flink 1.2.1 (RC1)
Posted by Stephan Ewen <se...@apache.org>.
+1 for options (1), but also invest the time to fix it properly for 1.2.2
On Mon, Apr 3, 2017 at 9:10 AM, Kostas Kloudas <k....@data-artisans.com>
wrote:
> +1 for 1
>
> > On Apr 3, 2017, at 5:52 PM, Till Rohrmann <tr...@apache.org> wrote:
> >
> > +1 for option 1)
> >
> > On Mon, Apr 3, 2017 at 5:48 PM, Fabian Hueske <fh...@gmail.com> wrote:
> >
> >> +1 to option 1)
> >>
> >> 2017-04-03 16:57 GMT+02:00 Ted Yu <yu...@gmail.com>:
> >>
> >>> Looks like #1 is better - 1.2.1 would be at least as stable as 1.2.0
> >>>
> >>> Cheers
> >>>
> >>> On Mon, Apr 3, 2017 at 7:39 AM, Aljoscha Krettek <al...@apache.org>
> >>> wrote:
> >>>
> >>>> Just so we’re all on the same page. ;-)
> >>>>
> >>>> There was https://issues.apache.org/jira/browse/FLINK-5808 which was
> a
> >>>> bug that we initially discovered in Flink 1.2 which was/is about
> >> missing
> >>>> verification for the correctness of the combination of parallelism and
> >>>> max-parallelism. Due to lacking test coverage this introduced two more
> >>> bugs:
> >>>> - https://issues.apache.org/jira/browse/FLINK-6188: Some
> >>>> setParallelism() methods can't cope with default parallelism
> >>>> - https://issues.apache.org/jira/browse/FLINK-6209:
> >>>> StreamPlanEnvironment always has a parallelism of 1
> >>>>
> >>>> IMHO, the options are:
> >>>> 1) revert the changes made for FLINK-5808 on the release-1.2 branch
> >> and
> >>>> live with the bug still being present
> >>>> 2) put in more work to fix FLINK-5808 which requires fixing some
> >>> problems
> >>>> that have existed for a long time with how the parallelism is set in
> >>>> streaming programs
> >>>>
> >>>> Best,
> >>>> Aljoscha
> >>>>
> >>>>> On 31. Mar 2017, at 21:34, Robert Metzger <rm...@apache.org>
> >> wrote:
> >>>>>
> >>>>> I don't know what is best to do, but I think releasing 1.2.1 with
> >>>>> potentially more bugs than 1.2.0 is not a good option.
> >>>>> I suspect a good workaround for FLINK-6188
> >>>>> <https://issues.apache.org/jira/browse/FLINK-6188> is setting the
> >>>>> parallelism manually for operators that can't cope with the default
> >> -1
> >>>>> parallelism.
> >>>>>
> >>>>> On Fri, Mar 31, 2017 at 9:06 PM, Aljoscha Krettek <
> >> aljoscha@apache.org
> >>>>
> >>>>> wrote:
> >>>>>
> >>>>>> You mean reverting the changes around FLINK-5808 [1]? This is what
> >>>>>> introduced the follow-up FLINK-6188 [2].
> >>>>>>
> >>>>>> [1] https://issues.apache.org/jira/browse/FLINK-5808
> >>>>>> [2]https://issues.apache.org/jira/browse/FLINK-6188
> >>>>>>
> >>>>>> On Fri, Mar 31, 2017, at 19:10, Robert Metzger wrote:
> >>>>>>> I think reverting FLINK-6188 for the 1.2 branch might be a good
> >> idea.
> >>>>>>> FLINK-6188 introduced two new bugs, so undoing the FLINK-6188 fix
> >>> will
> >>>>>>> lead
> >>>>>>> only to one known bug in 1.2.1, instead of an uncertain number of
> >>>> issues.
> >>>>>>> So 1.2.1 is not going to be worse than 1.2.0
> >>>>>>>
> >>>>>>> The fix will hopefully make it into 1.2.2 then.
> >>>>>>>
> >>>>>>> Any other thoughts on this?
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On Fri, Mar 31, 2017 at 6:46 PM, Fabian Hueske <fh...@gmail.com>
> >>>>>> wrote:
> >>>>>>>
> >>>>>>>> I merged the fix for FLINK-6044 to the release-1.2 and release-1.1
> >>>>>> branch.
> >>>>>>>>
> >>>>>>>> 2017-03-31 15:02 GMT+02:00 Fabian Hueske <fh...@gmail.com>:
> >>>>>>>>
> >>>>>>>>> We should also backport the fix for FLINK-6044 to Flink 1.2.1.
> >>>>>>>>>
> >>>>>>>>> I'll take care of that.
> >>>>>>>>>
> >>>>>>>>> 2017-03-30 18:50 GMT+02:00 Aljoscha Krettek <aljoscha@apache.org
> >>> :
> >>>>>>>>>
> >>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6188 turns out to
> >> be
> >>> a
> >>>>>> bit
> >>>>>>>>>> more involved, see my comments on the PR:
> >>>>>>>>>> https://github.com/apache/flink/pull/3616.
> >>>>>>>>>>
> >>>>>>>>>> As I said there, maybe we should revert the commits regarding
> >>>>>>>>>> parallelism/max-parallelism changes and release and then fix it
> >>>>>> later.
> >>>>>>>>>>
> >>>>>>>>>> On Wed, Mar 29, 2017, at 23:08, Aljoscha Krettek wrote:
> >>>>>>>>>>> I commented on FLINK-6214: I think it's working as intended,
> >>>>>> although
> >>>>>>>> we
> >>>>>>>>>>> could fix the javadoc/doc.
> >>>>>>>>>>>
> >>>>>>>>>>> On Wed, Mar 29, 2017, at 17:35, Timo Walther wrote:
> >>>>>>>>>>>> A user reported that all tumbling and slinding window
> >> assigners
> >>>>>>>>>> contain
> >>>>>>>>>>>> a pretty obvious bug about offsets.
> >>>>>>>>>>>>
> >>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6214
> >>>>>>>>>>>>
> >>>>>>>>>>>> I think we should also fix this for 1.2.1. What do you think?
> >>>>>>>>>>>>
> >>>>>>>>>>>> Regards,
> >>>>>>>>>>>> Timo
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> Am 29/03/17 um 11:30 schrieb Robert Metzger:
> >>>>>>>>>>>>> Hi Haohui,
> >>>>>>>>>>>>> I agree that we should fix the parallelism issue. Otherwise,
> >>>>>> the
> >>>>>>>>>> 1.2.1
> >>>>>>>>>>>>> release would introduce a new bug.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Tue, Mar 28, 2017 at 11:59 PM, Haohui Mai <
> >>>>>> ricetons@gmail.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> -1 (non-binding)
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> We recently found out that all jobs submitted via UI will
> >>>>>> have a
> >>>>>>>>>>>>>> parallelism of 1, potentially due to FLINK-5808.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Filed FLINK-6209 to track it.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> ~Haohui
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Mon, Mar 27, 2017 at 2:59 AM Chesnay Schepler <
> >>>>>>>>>> chesnay@apache.org>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> If possible I would like to include FLINK-6183 & FLINK-6184
> >>>>>> as
> >>>>>>>>>> well.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> They fix 2 metric-related issues that could arise when a
> >>>>>> Task is
> >>>>>>>>>>>>>>> cancelled very early. (like, right away)
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> FLINK-6183 fixes a memory leak where the TaskMetricGroup
> >> was
> >>>>>>>>>> never closed
> >>>>>>>>>>>>>>> FLINK-6184 fixes a NullPointerExceptions in the buffer
> >>>>>> metrics
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> PR here: https://github.com/apache/flink/pull/3611
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On 26.03.2017 12:35, Aljoscha Krettek wrote:
> >>>>>>>>>>>>>>>> I opened a PR for FLINK-6188: https://github.com/apache/
> >>>>>>>>>>>>>> flink/pull/3616
> >>>>>>>>>>>>>>> <https://github.com/apache/flink/pull/3616>
> >>>>>>>>>>>>>>>> This improves the previously very sparse test coverage for
> >>>>>>>>>>>>>>> timestamp/watermark assigners and fixes the bug.
> >>>>>>>>>>>>>>>>> On 25 Mar 2017, at 10:22, Ufuk Celebi <uc...@apache.org>
> >>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I agree with Aljoscha.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> -1 because of FLINK-6188
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Sat, Mar 25, 2017 at 9:38 AM, Aljoscha Krettek <
> >>>>>>>>>>>>>> aljoscha@apache.org>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>> I filed this issue, which was observed by a user:
> >>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6188
> >>>>>>>>>>>>>>>>>> I think that’s blocking for 1.2.1.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On 24 Mar 2017, at 18:57, Ufuk Celebi <uc...@apache.org>
> >>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> RC1 doesn't contain Stefan's backport for the
> >>>>>> Asynchronous
> >>>>>>>>>> snapshots
> >>>>>>>>>>>>>>>>>>> for heap-based keyed state that has been merged. Should
> >>>>>> we
> >>>>>>>>>> create
> >>>>>>>>>>>>>> RC2
> >>>>>>>>>>>>>>>>>>> with that fix since the voting period only starts on
> >>>>>> Monday?
> >>>>>>>>>> I think
> >>>>>>>>>>>>>>>>>>> it would only mean rerunning the scripts on your side,
> >>>>>>>> right?
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> – Ufuk
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Fri, Mar 24, 2017 at 3:05 PM, Robert Metzger <
> >>>>>>>>>>>>>> rmetzger@apache.org>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>> Dear Flink community,
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Please vote on releasing the following candidate as
> >>>>>> Apache
> >>>>>>>>>> Flink
> >>>>>>>>>>>>>>> version 1.2
> >>>>>>>>>>>>>>>>>>>> .1.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> The commit to be voted on:
> >>>>>>>>>>>>>>>>>>>> *732e55bd* (*
> >>>>>>>>>>>>>>> http://git-wip-us.apache.org/repos/asf/flink/commit/
> >>>>>> 732e55bd
> >>>>>>>>>>>>>>>>>>>> <http://git-wip-us.apache.org/
> >>>>>>>> repos/asf/flink/commit/732e55b
> >>>>>>>>>> d>*)
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Branch:
> >>>>>>>>>>>>>>>>>>>> release-1.2.1-rc1
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> The release artifacts to be voted on can be found at:
> >>>>>>>>>>>>>>>>>>>> *http://people.apache.org/~rmetzger/flink-1.2.1-rc1/
> >>>>>>>>>>>>>>>>>>>> <http://people.apache.org/~rmetzger/flink-1.2.1-rc1/
> >>> *
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> The release artifacts are signed with the key with
> >>>>>>>>>> fingerprint
> >>>>>>>>>>>>>>> D9839159:
> >>>>>>>>>>>>>>>>>>>> http://www.apache.org/dist/flink/KEYS
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> The staging repository for this release can be found
> >>>>>> at:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> https://repository.apache.org/
> >>>>>> content/repositories/orgapache
> >>>>>>>>>> flink-1116
> >>>>>>>>>>>>>>>>>>>> ------------------------------
> >>>>>>>> ------------------------------
> >>>>>>>>>> -
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> The vote ends on Wednesday, March 29, 2017, 3pm CET.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> [ ] +1 Release this package as Apache Flink 1.2.1
> >>>>>>>>>>>>>>>>>>>> [ ] -1 Do not release this package, because ...
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>
> >>>>
> >>>
> >>
>
>
Re: [VOTE] Release Apache Flink 1.2.1 (RC1)
Posted by Kostas Kloudas <k....@data-artisans.com>.
+1 for 1
> On Apr 3, 2017, at 5:52 PM, Till Rohrmann <tr...@apache.org> wrote:
>
> +1 for option 1)
>
> On Mon, Apr 3, 2017 at 5:48 PM, Fabian Hueske <fh...@gmail.com> wrote:
>
>> +1 to option 1)
>>
>> 2017-04-03 16:57 GMT+02:00 Ted Yu <yu...@gmail.com>:
>>
>>> Looks like #1 is better - 1.2.1 would be at least as stable as 1.2.0
>>>
>>> Cheers
>>>
>>> On Mon, Apr 3, 2017 at 7:39 AM, Aljoscha Krettek <al...@apache.org>
>>> wrote:
>>>
>>>> Just so we’re all on the same page. ;-)
>>>>
>>>> There was https://issues.apache.org/jira/browse/FLINK-5808 which was a
>>>> bug that we initially discovered in Flink 1.2 which was/is about
>> missing
>>>> verification for the correctness of the combination of parallelism and
>>>> max-parallelism. Due to lacking test coverage this introduced two more
>>> bugs:
>>>> - https://issues.apache.org/jira/browse/FLINK-6188: Some
>>>> setParallelism() methods can't cope with default parallelism
>>>> - https://issues.apache.org/jira/browse/FLINK-6209:
>>>> StreamPlanEnvironment always has a parallelism of 1
>>>>
>>>> IMHO, the options are:
>>>> 1) revert the changes made for FLINK-5808 on the release-1.2 branch
>> and
>>>> live with the bug still being present
>>>> 2) put in more work to fix FLINK-5808 which requires fixing some
>>> problems
>>>> that have existed for a long time with how the parallelism is set in
>>>> streaming programs
>>>>
>>>> Best,
>>>> Aljoscha
>>>>
>>>>> On 31. Mar 2017, at 21:34, Robert Metzger <rm...@apache.org>
>> wrote:
>>>>>
>>>>> I don't know what is best to do, but I think releasing 1.2.1 with
>>>>> potentially more bugs than 1.2.0 is not a good option.
>>>>> I suspect a good workaround for FLINK-6188
>>>>> <https://issues.apache.org/jira/browse/FLINK-6188> is setting the
>>>>> parallelism manually for operators that can't cope with the default
>> -1
>>>>> parallelism.
>>>>>
>>>>> On Fri, Mar 31, 2017 at 9:06 PM, Aljoscha Krettek <
>> aljoscha@apache.org
>>>>
>>>>> wrote:
>>>>>
>>>>>> You mean reverting the changes around FLINK-5808 [1]? This is what
>>>>>> introduced the follow-up FLINK-6188 [2].
>>>>>>
>>>>>> [1] https://issues.apache.org/jira/browse/FLINK-5808
>>>>>> [2]https://issues.apache.org/jira/browse/FLINK-6188
>>>>>>
>>>>>> On Fri, Mar 31, 2017, at 19:10, Robert Metzger wrote:
>>>>>>> I think reverting FLINK-6188 for the 1.2 branch might be a good
>> idea.
>>>>>>> FLINK-6188 introduced two new bugs, so undoing the FLINK-6188 fix
>>> will
>>>>>>> lead
>>>>>>> only to one known bug in 1.2.1, instead of an uncertain number of
>>>> issues.
>>>>>>> So 1.2.1 is not going to be worse than 1.2.0
>>>>>>>
>>>>>>> The fix will hopefully make it into 1.2.2 then.
>>>>>>>
>>>>>>> Any other thoughts on this?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Mar 31, 2017 at 6:46 PM, Fabian Hueske <fh...@gmail.com>
>>>>>> wrote:
>>>>>>>
>>>>>>>> I merged the fix for FLINK-6044 to the release-1.2 and release-1.1
>>>>>> branch.
>>>>>>>>
>>>>>>>> 2017-03-31 15:02 GMT+02:00 Fabian Hueske <fh...@gmail.com>:
>>>>>>>>
>>>>>>>>> We should also backport the fix for FLINK-6044 to Flink 1.2.1.
>>>>>>>>>
>>>>>>>>> I'll take care of that.
>>>>>>>>>
>>>>>>>>> 2017-03-30 18:50 GMT+02:00 Aljoscha Krettek <aljoscha@apache.org
>>> :
>>>>>>>>>
>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6188 turns out to
>> be
>>> a
>>>>>> bit
>>>>>>>>>> more involved, see my comments on the PR:
>>>>>>>>>> https://github.com/apache/flink/pull/3616.
>>>>>>>>>>
>>>>>>>>>> As I said there, maybe we should revert the commits regarding
>>>>>>>>>> parallelism/max-parallelism changes and release and then fix it
>>>>>> later.
>>>>>>>>>>
>>>>>>>>>> On Wed, Mar 29, 2017, at 23:08, Aljoscha Krettek wrote:
>>>>>>>>>>> I commented on FLINK-6214: I think it's working as intended,
>>>>>> although
>>>>>>>> we
>>>>>>>>>>> could fix the javadoc/doc.
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Mar 29, 2017, at 17:35, Timo Walther wrote:
>>>>>>>>>>>> A user reported that all tumbling and slinding window
>> assigners
>>>>>>>>>> contain
>>>>>>>>>>>> a pretty obvious bug about offsets.
>>>>>>>>>>>>
>>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6214
>>>>>>>>>>>>
>>>>>>>>>>>> I think we should also fix this for 1.2.1. What do you think?
>>>>>>>>>>>>
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Timo
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Am 29/03/17 um 11:30 schrieb Robert Metzger:
>>>>>>>>>>>>> Hi Haohui,
>>>>>>>>>>>>> I agree that we should fix the parallelism issue. Otherwise,
>>>>>> the
>>>>>>>>>> 1.2.1
>>>>>>>>>>>>> release would introduce a new bug.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Mar 28, 2017 at 11:59 PM, Haohui Mai <
>>>>>> ricetons@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> -1 (non-binding)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> We recently found out that all jobs submitted via UI will
>>>>>> have a
>>>>>>>>>>>>>> parallelism of 1, potentially due to FLINK-5808.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Filed FLINK-6209 to track it.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ~Haohui
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Mar 27, 2017 at 2:59 AM Chesnay Schepler <
>>>>>>>>>> chesnay@apache.org>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> If possible I would like to include FLINK-6183 & FLINK-6184
>>>>>> as
>>>>>>>>>> well.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> They fix 2 metric-related issues that could arise when a
>>>>>> Task is
>>>>>>>>>>>>>>> cancelled very early. (like, right away)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> FLINK-6183 fixes a memory leak where the TaskMetricGroup
>> was
>>>>>>>>>> never closed
>>>>>>>>>>>>>>> FLINK-6184 fixes a NullPointerExceptions in the buffer
>>>>>> metrics
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> PR here: https://github.com/apache/flink/pull/3611
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On 26.03.2017 12:35, Aljoscha Krettek wrote:
>>>>>>>>>>>>>>>> I opened a PR for FLINK-6188: https://github.com/apache/
>>>>>>>>>>>>>> flink/pull/3616
>>>>>>>>>>>>>>> <https://github.com/apache/flink/pull/3616>
>>>>>>>>>>>>>>>> This improves the previously very sparse test coverage for
>>>>>>>>>>>>>>> timestamp/watermark assigners and fixes the bug.
>>>>>>>>>>>>>>>>> On 25 Mar 2017, at 10:22, Ufuk Celebi <uc...@apache.org>
>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I agree with Aljoscha.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> -1 because of FLINK-6188
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Sat, Mar 25, 2017 at 9:38 AM, Aljoscha Krettek <
>>>>>>>>>>>>>> aljoscha@apache.org>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>> I filed this issue, which was observed by a user:
>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6188
>>>>>>>>>>>>>>>>>> I think that’s blocking for 1.2.1.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On 24 Mar 2017, at 18:57, Ufuk Celebi <uc...@apache.org>
>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> RC1 doesn't contain Stefan's backport for the
>>>>>> Asynchronous
>>>>>>>>>> snapshots
>>>>>>>>>>>>>>>>>>> for heap-based keyed state that has been merged. Should
>>>>>> we
>>>>>>>>>> create
>>>>>>>>>>>>>> RC2
>>>>>>>>>>>>>>>>>>> with that fix since the voting period only starts on
>>>>>> Monday?
>>>>>>>>>> I think
>>>>>>>>>>>>>>>>>>> it would only mean rerunning the scripts on your side,
>>>>>>>> right?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> – Ufuk
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Fri, Mar 24, 2017 at 3:05 PM, Robert Metzger <
>>>>>>>>>>>>>> rmetzger@apache.org>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>> Dear Flink community,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Please vote on releasing the following candidate as
>>>>>> Apache
>>>>>>>>>> Flink
>>>>>>>>>>>>>>> version 1.2
>>>>>>>>>>>>>>>>>>>> .1.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> The commit to be voted on:
>>>>>>>>>>>>>>>>>>>> *732e55bd* (*
>>>>>>>>>>>>>>> http://git-wip-us.apache.org/repos/asf/flink/commit/
>>>>>> 732e55bd
>>>>>>>>>>>>>>>>>>>> <http://git-wip-us.apache.org/
>>>>>>>> repos/asf/flink/commit/732e55b
>>>>>>>>>> d>*)
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Branch:
>>>>>>>>>>>>>>>>>>>> release-1.2.1-rc1
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> The release artifacts to be voted on can be found at:
>>>>>>>>>>>>>>>>>>>> *http://people.apache.org/~rmetzger/flink-1.2.1-rc1/
>>>>>>>>>>>>>>>>>>>> <http://people.apache.org/~rmetzger/flink-1.2.1-rc1/
>>> *
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> The release artifacts are signed with the key with
>>>>>>>>>> fingerprint
>>>>>>>>>>>>>>> D9839159:
>>>>>>>>>>>>>>>>>>>> http://www.apache.org/dist/flink/KEYS
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> The staging repository for this release can be found
>>>>>> at:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> https://repository.apache.org/
>>>>>> content/repositories/orgapache
>>>>>>>>>> flink-1116
>>>>>>>>>>>>>>>>>>>> ------------------------------
>>>>>>>> ------------------------------
>>>>>>>>>> -
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> The vote ends on Wednesday, March 29, 2017, 3pm CET.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> [ ] +1 Release this package as Apache Flink 1.2.1
>>>>>>>>>>>>>>>>>>>> [ ] -1 Do not release this package, because ...
>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>>>>
>>>
>>
Re: [VOTE] Release Apache Flink 1.2.1 (RC1)
Posted by Till Rohrmann <tr...@apache.org>.
+1 for option 1)
On Mon, Apr 3, 2017 at 5:48 PM, Fabian Hueske <fh...@gmail.com> wrote:
> +1 to option 1)
>
> 2017-04-03 16:57 GMT+02:00 Ted Yu <yu...@gmail.com>:
>
> > Looks like #1 is better - 1.2.1 would be at least as stable as 1.2.0
> >
> > Cheers
> >
> > On Mon, Apr 3, 2017 at 7:39 AM, Aljoscha Krettek <al...@apache.org>
> > wrote:
> >
> > > Just so we’re all on the same page. ;-)
> > >
> > > There was https://issues.apache.org/jira/browse/FLINK-5808 which was a
> > > bug that we initially discovered in Flink 1.2 which was/is about
> missing
> > > verification for the correctness of the combination of parallelism and
> > > max-parallelism. Due to lacking test coverage this introduced two more
> > bugs:
> > > - https://issues.apache.org/jira/browse/FLINK-6188: Some
> > > setParallelism() methods can't cope with default parallelism
> > > - https://issues.apache.org/jira/browse/FLINK-6209:
> > > StreamPlanEnvironment always has a parallelism of 1
> > >
> > > IMHO, the options are:
> > > 1) revert the changes made for FLINK-5808 on the release-1.2 branch
> and
> > > live with the bug still being present
> > > 2) put in more work to fix FLINK-5808 which requires fixing some
> > problems
> > > that have existed for a long time with how the parallelism is set in
> > > streaming programs
> > >
> > > Best,
> > > Aljoscha
> > >
> > > > On 31. Mar 2017, at 21:34, Robert Metzger <rm...@apache.org>
> wrote:
> > > >
> > > > I don't know what is best to do, but I think releasing 1.2.1 with
> > > > potentially more bugs than 1.2.0 is not a good option.
> > > > I suspect a good workaround for FLINK-6188
> > > > <https://issues.apache.org/jira/browse/FLINK-6188> is setting the
> > > > parallelism manually for operators that can't cope with the default
> -1
> > > > parallelism.
> > > >
> > > > On Fri, Mar 31, 2017 at 9:06 PM, Aljoscha Krettek <
> aljoscha@apache.org
> > >
> > > > wrote:
> > > >
> > > >> You mean reverting the changes around FLINK-5808 [1]? This is what
> > > >> introduced the follow-up FLINK-6188 [2].
> > > >>
> > > >> [1] https://issues.apache.org/jira/browse/FLINK-5808
> > > >> [2]https://issues.apache.org/jira/browse/FLINK-6188
> > > >>
> > > >> On Fri, Mar 31, 2017, at 19:10, Robert Metzger wrote:
> > > >>> I think reverting FLINK-6188 for the 1.2 branch might be a good
> idea.
> > > >>> FLINK-6188 introduced two new bugs, so undoing the FLINK-6188 fix
> > will
> > > >>> lead
> > > >>> only to one known bug in 1.2.1, instead of an uncertain number of
> > > issues.
> > > >>> So 1.2.1 is not going to be worse than 1.2.0
> > > >>>
> > > >>> The fix will hopefully make it into 1.2.2 then.
> > > >>>
> > > >>> Any other thoughts on this?
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>> On Fri, Mar 31, 2017 at 6:46 PM, Fabian Hueske <fh...@gmail.com>
> > > >> wrote:
> > > >>>
> > > >>>> I merged the fix for FLINK-6044 to the release-1.2 and release-1.1
> > > >> branch.
> > > >>>>
> > > >>>> 2017-03-31 15:02 GMT+02:00 Fabian Hueske <fh...@gmail.com>:
> > > >>>>
> > > >>>>> We should also backport the fix for FLINK-6044 to Flink 1.2.1.
> > > >>>>>
> > > >>>>> I'll take care of that.
> > > >>>>>
> > > >>>>> 2017-03-30 18:50 GMT+02:00 Aljoscha Krettek <aljoscha@apache.org
> >:
> > > >>>>>
> > > >>>>>> https://issues.apache.org/jira/browse/FLINK-6188 turns out to
> be
> > a
> > > >> bit
> > > >>>>>> more involved, see my comments on the PR:
> > > >>>>>> https://github.com/apache/flink/pull/3616.
> > > >>>>>>
> > > >>>>>> As I said there, maybe we should revert the commits regarding
> > > >>>>>> parallelism/max-parallelism changes and release and then fix it
> > > >> later.
> > > >>>>>>
> > > >>>>>> On Wed, Mar 29, 2017, at 23:08, Aljoscha Krettek wrote:
> > > >>>>>>> I commented on FLINK-6214: I think it's working as intended,
> > > >> although
> > > >>>> we
> > > >>>>>>> could fix the javadoc/doc.
> > > >>>>>>>
> > > >>>>>>> On Wed, Mar 29, 2017, at 17:35, Timo Walther wrote:
> > > >>>>>>>> A user reported that all tumbling and slinding window
> assigners
> > > >>>>>> contain
> > > >>>>>>>> a pretty obvious bug about offsets.
> > > >>>>>>>>
> > > >>>>>>>> https://issues.apache.org/jira/browse/FLINK-6214
> > > >>>>>>>>
> > > >>>>>>>> I think we should also fix this for 1.2.1. What do you think?
> > > >>>>>>>>
> > > >>>>>>>> Regards,
> > > >>>>>>>> Timo
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> Am 29/03/17 um 11:30 schrieb Robert Metzger:
> > > >>>>>>>>> Hi Haohui,
> > > >>>>>>>>> I agree that we should fix the parallelism issue. Otherwise,
> > > >> the
> > > >>>>>> 1.2.1
> > > >>>>>>>>> release would introduce a new bug.
> > > >>>>>>>>>
> > > >>>>>>>>> On Tue, Mar 28, 2017 at 11:59 PM, Haohui Mai <
> > > >> ricetons@gmail.com>
> > > >>>>>> wrote:
> > > >>>>>>>>>
> > > >>>>>>>>>> -1 (non-binding)
> > > >>>>>>>>>>
> > > >>>>>>>>>> We recently found out that all jobs submitted via UI will
> > > >> have a
> > > >>>>>>>>>> parallelism of 1, potentially due to FLINK-5808.
> > > >>>>>>>>>>
> > > >>>>>>>>>> Filed FLINK-6209 to track it.
> > > >>>>>>>>>>
> > > >>>>>>>>>> ~Haohui
> > > >>>>>>>>>>
> > > >>>>>>>>>> On Mon, Mar 27, 2017 at 2:59 AM Chesnay Schepler <
> > > >>>>>> chesnay@apache.org>
> > > >>>>>>>>>> wrote:
> > > >>>>>>>>>>
> > > >>>>>>>>>>> If possible I would like to include FLINK-6183 & FLINK-6184
> > > >> as
> > > >>>>>> well.
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> They fix 2 metric-related issues that could arise when a
> > > >> Task is
> > > >>>>>>>>>>> cancelled very early. (like, right away)
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> FLINK-6183 fixes a memory leak where the TaskMetricGroup
> was
> > > >>>>>> never closed
> > > >>>>>>>>>>> FLINK-6184 fixes a NullPointerExceptions in the buffer
> > > >> metrics
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> PR here: https://github.com/apache/flink/pull/3611
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> On 26.03.2017 12:35, Aljoscha Krettek wrote:
> > > >>>>>>>>>>>> I opened a PR for FLINK-6188: https://github.com/apache/
> > > >>>>>>>>>> flink/pull/3616
> > > >>>>>>>>>>> <https://github.com/apache/flink/pull/3616>
> > > >>>>>>>>>>>> This improves the previously very sparse test coverage for
> > > >>>>>>>>>>> timestamp/watermark assigners and fixes the bug.
> > > >>>>>>>>>>>>> On 25 Mar 2017, at 10:22, Ufuk Celebi <uc...@apache.org>
> > > >> wrote:
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> I agree with Aljoscha.
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> -1 because of FLINK-6188
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> On Sat, Mar 25, 2017 at 9:38 AM, Aljoscha Krettek <
> > > >>>>>>>>>> aljoscha@apache.org>
> > > >>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>> I filed this issue, which was observed by a user:
> > > >>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6188
> > > >>>>>>>>>>>>>> I think that’s blocking for 1.2.1.
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> On 24 Mar 2017, at 18:57, Ufuk Celebi <uc...@apache.org>
> > > >>>> wrote:
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> RC1 doesn't contain Stefan's backport for the
> > > >> Asynchronous
> > > >>>>>> snapshots
> > > >>>>>>>>>>>>>>> for heap-based keyed state that has been merged. Should
> > > >> we
> > > >>>>>> create
> > > >>>>>>>>>> RC2
> > > >>>>>>>>>>>>>>> with that fix since the voting period only starts on
> > > >> Monday?
> > > >>>>>> I think
> > > >>>>>>>>>>>>>>> it would only mean rerunning the scripts on your side,
> > > >>>> right?
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> – Ufuk
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> On Fri, Mar 24, 2017 at 3:05 PM, Robert Metzger <
> > > >>>>>>>>>> rmetzger@apache.org>
> > > >>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>> Dear Flink community,
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> Please vote on releasing the following candidate as
> > > >> Apache
> > > >>>>>> Flink
> > > >>>>>>>>>>> version 1.2
> > > >>>>>>>>>>>>>>>> .1.
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> The commit to be voted on:
> > > >>>>>>>>>>>>>>>> *732e55bd* (*
> > > >>>>>>>>>>> http://git-wip-us.apache.org/repos/asf/flink/commit/
> > > >> 732e55bd
> > > >>>>>>>>>>>>>>>> <http://git-wip-us.apache.org/
> > > >>>> repos/asf/flink/commit/732e55b
> > > >>>>>> d>*)
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> Branch:
> > > >>>>>>>>>>>>>>>> release-1.2.1-rc1
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> The release artifacts to be voted on can be found at:
> > > >>>>>>>>>>>>>>>> *http://people.apache.org/~rmetzger/flink-1.2.1-rc1/
> > > >>>>>>>>>>>>>>>> <http://people.apache.org/~rmetzger/flink-1.2.1-rc1/
> >*
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> The release artifacts are signed with the key with
> > > >>>>>> fingerprint
> > > >>>>>>>>>>> D9839159:
> > > >>>>>>>>>>>>>>>> http://www.apache.org/dist/flink/KEYS
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> The staging repository for this release can be found
> > > >> at:
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>> https://repository.apache.org/
> > > >> content/repositories/orgapache
> > > >>>>>> flink-1116
> > > >>>>>>>>>>>>>>>> ------------------------------
> > > >>>> ------------------------------
> > > >>>>>> -
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> The vote ends on Wednesday, March 29, 2017, 3pm CET.
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> [ ] +1 Release this package as Apache Flink 1.2.1
> > > >>>>>>>>>>>>>>>> [ ] -1 Do not release this package, because ...
> > > >>>>>>>>>>>
> > > >>>>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>>>
> > > >>>>
> > > >>
> > >
> > >
> >
>
Re: [VOTE] Release Apache Flink 1.2.1 (RC1)
Posted by Fabian Hueske <fh...@gmail.com>.
+1 to option 1)
2017-04-03 16:57 GMT+02:00 Ted Yu <yu...@gmail.com>:
> Looks like #1 is better - 1.2.1 would be at least as stable as 1.2.0
>
> Cheers
>
> On Mon, Apr 3, 2017 at 7:39 AM, Aljoscha Krettek <al...@apache.org>
> wrote:
>
> > Just so we’re all on the same page. ;-)
> >
> > There was https://issues.apache.org/jira/browse/FLINK-5808 which was a
> > bug that we initially discovered in Flink 1.2 which was/is about missing
> > verification for the correctness of the combination of parallelism and
> > max-parallelism. Due to lacking test coverage this introduced two more
> bugs:
> > - https://issues.apache.org/jira/browse/FLINK-6188: Some
> > setParallelism() methods can't cope with default parallelism
> > - https://issues.apache.org/jira/browse/FLINK-6209:
> > StreamPlanEnvironment always has a parallelism of 1
> >
> > IMHO, the options are:
> > 1) revert the changes made for FLINK-5808 on the release-1.2 branch and
> > live with the bug still being present
> > 2) put in more work to fix FLINK-5808 which requires fixing some
> problems
> > that have existed for a long time with how the parallelism is set in
> > streaming programs
> >
> > Best,
> > Aljoscha
> >
> > > On 31. Mar 2017, at 21:34, Robert Metzger <rm...@apache.org> wrote:
> > >
> > > I don't know what is best to do, but I think releasing 1.2.1 with
> > > potentially more bugs than 1.2.0 is not a good option.
> > > I suspect a good workaround for FLINK-6188
> > > <https://issues.apache.org/jira/browse/FLINK-6188> is setting the
> > > parallelism manually for operators that can't cope with the default -1
> > > parallelism.
> > >
> > > On Fri, Mar 31, 2017 at 9:06 PM, Aljoscha Krettek <aljoscha@apache.org
> >
> > > wrote:
> > >
> > >> You mean reverting the changes around FLINK-5808 [1]? This is what
> > >> introduced the follow-up FLINK-6188 [2].
> > >>
> > >> [1] https://issues.apache.org/jira/browse/FLINK-5808
> > >> [2]https://issues.apache.org/jira/browse/FLINK-6188
> > >>
> > >> On Fri, Mar 31, 2017, at 19:10, Robert Metzger wrote:
> > >>> I think reverting FLINK-6188 for the 1.2 branch might be a good idea.
> > >>> FLINK-6188 introduced two new bugs, so undoing the FLINK-6188 fix
> will
> > >>> lead
> > >>> only to one known bug in 1.2.1, instead of an uncertain number of
> > issues.
> > >>> So 1.2.1 is not going to be worse than 1.2.0
> > >>>
> > >>> The fix will hopefully make it into 1.2.2 then.
> > >>>
> > >>> Any other thoughts on this?
> > >>>
> > >>>
> > >>>
> > >>>
> > >>> On Fri, Mar 31, 2017 at 6:46 PM, Fabian Hueske <fh...@gmail.com>
> > >> wrote:
> > >>>
> > >>>> I merged the fix for FLINK-6044 to the release-1.2 and release-1.1
> > >> branch.
> > >>>>
> > >>>> 2017-03-31 15:02 GMT+02:00 Fabian Hueske <fh...@gmail.com>:
> > >>>>
> > >>>>> We should also backport the fix for FLINK-6044 to Flink 1.2.1.
> > >>>>>
> > >>>>> I'll take care of that.
> > >>>>>
> > >>>>> 2017-03-30 18:50 GMT+02:00 Aljoscha Krettek <al...@apache.org>:
> > >>>>>
> > >>>>>> https://issues.apache.org/jira/browse/FLINK-6188 turns out to be
> a
> > >> bit
> > >>>>>> more involved, see my comments on the PR:
> > >>>>>> https://github.com/apache/flink/pull/3616.
> > >>>>>>
> > >>>>>> As I said there, maybe we should revert the commits regarding
> > >>>>>> parallelism/max-parallelism changes and release and then fix it
> > >> later.
> > >>>>>>
> > >>>>>> On Wed, Mar 29, 2017, at 23:08, Aljoscha Krettek wrote:
> > >>>>>>> I commented on FLINK-6214: I think it's working as intended,
> > >> although
> > >>>> we
> > >>>>>>> could fix the javadoc/doc.
> > >>>>>>>
> > >>>>>>> On Wed, Mar 29, 2017, at 17:35, Timo Walther wrote:
> > >>>>>>>> A user reported that all tumbling and slinding window assigners
> > >>>>>> contain
> > >>>>>>>> a pretty obvious bug about offsets.
> > >>>>>>>>
> > >>>>>>>> https://issues.apache.org/jira/browse/FLINK-6214
> > >>>>>>>>
> > >>>>>>>> I think we should also fix this for 1.2.1. What do you think?
> > >>>>>>>>
> > >>>>>>>> Regards,
> > >>>>>>>> Timo
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> Am 29/03/17 um 11:30 schrieb Robert Metzger:
> > >>>>>>>>> Hi Haohui,
> > >>>>>>>>> I agree that we should fix the parallelism issue. Otherwise,
> > >> the
> > >>>>>> 1.2.1
> > >>>>>>>>> release would introduce a new bug.
> > >>>>>>>>>
> > >>>>>>>>> On Tue, Mar 28, 2017 at 11:59 PM, Haohui Mai <
> > >> ricetons@gmail.com>
> > >>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> -1 (non-binding)
> > >>>>>>>>>>
> > >>>>>>>>>> We recently found out that all jobs submitted via UI will
> > >> have a
> > >>>>>>>>>> parallelism of 1, potentially due to FLINK-5808.
> > >>>>>>>>>>
> > >>>>>>>>>> Filed FLINK-6209 to track it.
> > >>>>>>>>>>
> > >>>>>>>>>> ~Haohui
> > >>>>>>>>>>
> > >>>>>>>>>> On Mon, Mar 27, 2017 at 2:59 AM Chesnay Schepler <
> > >>>>>> chesnay@apache.org>
> > >>>>>>>>>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>>> If possible I would like to include FLINK-6183 & FLINK-6184
> > >> as
> > >>>>>> well.
> > >>>>>>>>>>>
> > >>>>>>>>>>> They fix 2 metric-related issues that could arise when a
> > >> Task is
> > >>>>>>>>>>> cancelled very early. (like, right away)
> > >>>>>>>>>>>
> > >>>>>>>>>>> FLINK-6183 fixes a memory leak where the TaskMetricGroup was
> > >>>>>> never closed
> > >>>>>>>>>>> FLINK-6184 fixes a NullPointerExceptions in the buffer
> > >> metrics
> > >>>>>>>>>>>
> > >>>>>>>>>>> PR here: https://github.com/apache/flink/pull/3611
> > >>>>>>>>>>>
> > >>>>>>>>>>> On 26.03.2017 12:35, Aljoscha Krettek wrote:
> > >>>>>>>>>>>> I opened a PR for FLINK-6188: https://github.com/apache/
> > >>>>>>>>>> flink/pull/3616
> > >>>>>>>>>>> <https://github.com/apache/flink/pull/3616>
> > >>>>>>>>>>>> This improves the previously very sparse test coverage for
> > >>>>>>>>>>> timestamp/watermark assigners and fixes the bug.
> > >>>>>>>>>>>>> On 25 Mar 2017, at 10:22, Ufuk Celebi <uc...@apache.org>
> > >> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> I agree with Aljoscha.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> -1 because of FLINK-6188
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> On Sat, Mar 25, 2017 at 9:38 AM, Aljoscha Krettek <
> > >>>>>>>>>> aljoscha@apache.org>
> > >>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>> I filed this issue, which was observed by a user:
> > >>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6188
> > >>>>>>>>>>>>>> I think that’s blocking for 1.2.1.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> On 24 Mar 2017, at 18:57, Ufuk Celebi <uc...@apache.org>
> > >>>> wrote:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> RC1 doesn't contain Stefan's backport for the
> > >> Asynchronous
> > >>>>>> snapshots
> > >>>>>>>>>>>>>>> for heap-based keyed state that has been merged. Should
> > >> we
> > >>>>>> create
> > >>>>>>>>>> RC2
> > >>>>>>>>>>>>>>> with that fix since the voting period only starts on
> > >> Monday?
> > >>>>>> I think
> > >>>>>>>>>>>>>>> it would only mean rerunning the scripts on your side,
> > >>>> right?
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> – Ufuk
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> On Fri, Mar 24, 2017 at 3:05 PM, Robert Metzger <
> > >>>>>>>>>> rmetzger@apache.org>
> > >>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>> Dear Flink community,
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Please vote on releasing the following candidate as
> > >> Apache
> > >>>>>> Flink
> > >>>>>>>>>>> version 1.2
> > >>>>>>>>>>>>>>>> .1.
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> The commit to be voted on:
> > >>>>>>>>>>>>>>>> *732e55bd* (*
> > >>>>>>>>>>> http://git-wip-us.apache.org/repos/asf/flink/commit/
> > >> 732e55bd
> > >>>>>>>>>>>>>>>> <http://git-wip-us.apache.org/
> > >>>> repos/asf/flink/commit/732e55b
> > >>>>>> d>*)
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Branch:
> > >>>>>>>>>>>>>>>> release-1.2.1-rc1
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> The release artifacts to be voted on can be found at:
> > >>>>>>>>>>>>>>>> *http://people.apache.org/~rmetzger/flink-1.2.1-rc1/
> > >>>>>>>>>>>>>>>> <http://people.apache.org/~rmetzger/flink-1.2.1-rc1/>*
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> The release artifacts are signed with the key with
> > >>>>>> fingerprint
> > >>>>>>>>>>> D9839159:
> > >>>>>>>>>>>>>>>> http://www.apache.org/dist/flink/KEYS
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> The staging repository for this release can be found
> > >> at:
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>> https://repository.apache.org/
> > >> content/repositories/orgapache
> > >>>>>> flink-1116
> > >>>>>>>>>>>>>>>> ------------------------------
> > >>>> ------------------------------
> > >>>>>> -
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> The vote ends on Wednesday, March 29, 2017, 3pm CET.
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> [ ] +1 Release this package as Apache Flink 1.2.1
> > >>>>>>>>>>>>>>>> [ ] -1 Do not release this package, because ...
> > >>>>>>>>>>>
> > >>>>>>>>
> > >>>>>>
> > >>>>>
> > >>>>>
> > >>>>
> > >>
> >
> >
>
Re: [VOTE] Release Apache Flink 1.2.1 (RC1)
Posted by Ted Yu <yu...@gmail.com>.
Looks like #1 is better - 1.2.1 would be at least as stable as 1.2.0
Cheers
On Mon, Apr 3, 2017 at 7:39 AM, Aljoscha Krettek <al...@apache.org>
wrote:
> Just so we’re all on the same page. ;-)
>
> There was https://issues.apache.org/jira/browse/FLINK-5808 which was a
> bug that we initially discovered in Flink 1.2 which was/is about missing
> verification for the correctness of the combination of parallelism and
> max-parallelism. Due to lacking test coverage this introduced two more bugs:
> - https://issues.apache.org/jira/browse/FLINK-6188: Some
> setParallelism() methods can't cope with default parallelism
> - https://issues.apache.org/jira/browse/FLINK-6209:
> StreamPlanEnvironment always has a parallelism of 1
>
> IMHO, the options are:
> 1) revert the changes made for FLINK-5808 on the release-1.2 branch and
> live with the bug still being present
> 2) put in more work to fix FLINK-5808 which requires fixing some problems
> that have existed for a long time with how the parallelism is set in
> streaming programs
>
> Best,
> Aljoscha
>
> > On 31. Mar 2017, at 21:34, Robert Metzger <rm...@apache.org> wrote:
> >
> > I don't know what is best to do, but I think releasing 1.2.1 with
> > potentially more bugs than 1.2.0 is not a good option.
> > I suspect a good workaround for FLINK-6188
> > <https://issues.apache.org/jira/browse/FLINK-6188> is setting the
> > parallelism manually for operators that can't cope with the default -1
> > parallelism.
> >
> > On Fri, Mar 31, 2017 at 9:06 PM, Aljoscha Krettek <al...@apache.org>
> > wrote:
> >
> >> You mean reverting the changes around FLINK-5808 [1]? This is what
> >> introduced the follow-up FLINK-6188 [2].
> >>
> >> [1] https://issues.apache.org/jira/browse/FLINK-5808
> >> [2]https://issues.apache.org/jira/browse/FLINK-6188
> >>
> >> On Fri, Mar 31, 2017, at 19:10, Robert Metzger wrote:
> >>> I think reverting FLINK-6188 for the 1.2 branch might be a good idea.
> >>> FLINK-6188 introduced two new bugs, so undoing the FLINK-6188 fix will
> >>> lead
> >>> only to one known bug in 1.2.1, instead of an uncertain number of
> issues.
> >>> So 1.2.1 is not going to be worse than 1.2.0
> >>>
> >>> The fix will hopefully make it into 1.2.2 then.
> >>>
> >>> Any other thoughts on this?
> >>>
> >>>
> >>>
> >>>
> >>> On Fri, Mar 31, 2017 at 6:46 PM, Fabian Hueske <fh...@gmail.com>
> >> wrote:
> >>>
> >>>> I merged the fix for FLINK-6044 to the release-1.2 and release-1.1
> >> branch.
> >>>>
> >>>> 2017-03-31 15:02 GMT+02:00 Fabian Hueske <fh...@gmail.com>:
> >>>>
> >>>>> We should also backport the fix for FLINK-6044 to Flink 1.2.1.
> >>>>>
> >>>>> I'll take care of that.
> >>>>>
> >>>>> 2017-03-30 18:50 GMT+02:00 Aljoscha Krettek <al...@apache.org>:
> >>>>>
> >>>>>> https://issues.apache.org/jira/browse/FLINK-6188 turns out to be a
> >> bit
> >>>>>> more involved, see my comments on the PR:
> >>>>>> https://github.com/apache/flink/pull/3616.
> >>>>>>
> >>>>>> As I said there, maybe we should revert the commits regarding
> >>>>>> parallelism/max-parallelism changes and release and then fix it
> >> later.
> >>>>>>
> >>>>>> On Wed, Mar 29, 2017, at 23:08, Aljoscha Krettek wrote:
> >>>>>>> I commented on FLINK-6214: I think it's working as intended,
> >> although
> >>>> we
> >>>>>>> could fix the javadoc/doc.
> >>>>>>>
> >>>>>>> On Wed, Mar 29, 2017, at 17:35, Timo Walther wrote:
> >>>>>>>> A user reported that all tumbling and slinding window assigners
> >>>>>> contain
> >>>>>>>> a pretty obvious bug about offsets.
> >>>>>>>>
> >>>>>>>> https://issues.apache.org/jira/browse/FLINK-6214
> >>>>>>>>
> >>>>>>>> I think we should also fix this for 1.2.1. What do you think?
> >>>>>>>>
> >>>>>>>> Regards,
> >>>>>>>> Timo
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Am 29/03/17 um 11:30 schrieb Robert Metzger:
> >>>>>>>>> Hi Haohui,
> >>>>>>>>> I agree that we should fix the parallelism issue. Otherwise,
> >> the
> >>>>>> 1.2.1
> >>>>>>>>> release would introduce a new bug.
> >>>>>>>>>
> >>>>>>>>> On Tue, Mar 28, 2017 at 11:59 PM, Haohui Mai <
> >> ricetons@gmail.com>
> >>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> -1 (non-binding)
> >>>>>>>>>>
> >>>>>>>>>> We recently found out that all jobs submitted via UI will
> >> have a
> >>>>>>>>>> parallelism of 1, potentially due to FLINK-5808.
> >>>>>>>>>>
> >>>>>>>>>> Filed FLINK-6209 to track it.
> >>>>>>>>>>
> >>>>>>>>>> ~Haohui
> >>>>>>>>>>
> >>>>>>>>>> On Mon, Mar 27, 2017 at 2:59 AM Chesnay Schepler <
> >>>>>> chesnay@apache.org>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> If possible I would like to include FLINK-6183 & FLINK-6184
> >> as
> >>>>>> well.
> >>>>>>>>>>>
> >>>>>>>>>>> They fix 2 metric-related issues that could arise when a
> >> Task is
> >>>>>>>>>>> cancelled very early. (like, right away)
> >>>>>>>>>>>
> >>>>>>>>>>> FLINK-6183 fixes a memory leak where the TaskMetricGroup was
> >>>>>> never closed
> >>>>>>>>>>> FLINK-6184 fixes a NullPointerExceptions in the buffer
> >> metrics
> >>>>>>>>>>>
> >>>>>>>>>>> PR here: https://github.com/apache/flink/pull/3611
> >>>>>>>>>>>
> >>>>>>>>>>> On 26.03.2017 12:35, Aljoscha Krettek wrote:
> >>>>>>>>>>>> I opened a PR for FLINK-6188: https://github.com/apache/
> >>>>>>>>>> flink/pull/3616
> >>>>>>>>>>> <https://github.com/apache/flink/pull/3616>
> >>>>>>>>>>>> This improves the previously very sparse test coverage for
> >>>>>>>>>>> timestamp/watermark assigners and fixes the bug.
> >>>>>>>>>>>>> On 25 Mar 2017, at 10:22, Ufuk Celebi <uc...@apache.org>
> >> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I agree with Aljoscha.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> -1 because of FLINK-6188
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Sat, Mar 25, 2017 at 9:38 AM, Aljoscha Krettek <
> >>>>>>>>>> aljoscha@apache.org>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>>> I filed this issue, which was observed by a user:
> >>>>>>>>>>> https://issues.apache.org/jira/browse/FLINK-6188
> >>>>>>>>>>>>>> I think that’s blocking for 1.2.1.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On 24 Mar 2017, at 18:57, Ufuk Celebi <uc...@apache.org>
> >>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> RC1 doesn't contain Stefan's backport for the
> >> Asynchronous
> >>>>>> snapshots
> >>>>>>>>>>>>>>> for heap-based keyed state that has been merged. Should
> >> we
> >>>>>> create
> >>>>>>>>>> RC2
> >>>>>>>>>>>>>>> with that fix since the voting period only starts on
> >> Monday?
> >>>>>> I think
> >>>>>>>>>>>>>>> it would only mean rerunning the scripts on your side,
> >>>> right?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> – Ufuk
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Fri, Mar 24, 2017 at 3:05 PM, Robert Metzger <
> >>>>>>>>>> rmetzger@apache.org>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>> Dear Flink community,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Please vote on releasing the following candidate as
> >> Apache
> >>>>>> Flink
> >>>>>>>>>>> version 1.2
> >>>>>>>>>>>>>>>> .1.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> The commit to be voted on:
> >>>>>>>>>>>>>>>> *732e55bd* (*
> >>>>>>>>>>> http://git-wip-us.apache.org/repos/asf/flink/commit/
> >> 732e55bd
> >>>>>>>>>>>>>>>> <http://git-wip-us.apache.org/
> >>>> repos/asf/flink/commit/732e55b
> >>>>>> d>*)
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Branch:
> >>>>>>>>>>>>>>>> release-1.2.1-rc1
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> The release artifacts to be voted on can be found at:
> >>>>>>>>>>>>>>>> *http://people.apache.org/~rmetzger/flink-1.2.1-rc1/
> >>>>>>>>>>>>>>>> <http://people.apache.org/~rmetzger/flink-1.2.1-rc1/>*
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> The release artifacts are signed with the key with
> >>>>>> fingerprint
> >>>>>>>>>>> D9839159:
> >>>>>>>>>>>>>>>> http://www.apache.org/dist/flink/KEYS
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> The staging repository for this release can be found
> >> at:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>> https://repository.apache.org/
> >> content/repositories/orgapache
> >>>>>> flink-1116
> >>>>>>>>>>>>>>>> ------------------------------
> >>>> ------------------------------
> >>>>>> -
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> The vote ends on Wednesday, March 29, 2017, 3pm CET.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> [ ] +1 Release this package as Apache Flink 1.2.1
> >>>>>>>>>>>>>>>> [ ] -1 Do not release this package, because ...
> >>>>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>
>
>