You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Stephan Ewen <se...@apache.org> on 2017/07/28 14:03:33 UTC
Re: [POLL] Dropping savepoint format compatibility for 1.1.x in the
Flink 1.4.0 release
Seems like no one raised a concern so far about dropping the savepoint
format compatibility for 1.1 in 1.4.
Leaving this thread open for some more days, but from the sentiment, it
seems like we should go ahead?
On Wed, Jul 12, 2017 at 4:43 PM, Stephan Ewen <se...@apache.org> wrote:
> Hi users!
>
> Flink currently maintains backwards compatibility for savepoint formats,
> which means that savepoints taken with Flink version 1.1.x and 1.2.x can be
> resumed in Flink 1.3.x
>
> We are discussing how many versions back to support. The proposition is
> the following:
>
> * Suggestion: Flink 1.4.0 will be able to resume savepoints taken with
> version 1.3.x and 1.2.x, but not savepoints from version 1.1.x and 1.0.x*
>
>
> The reason for that is that there is a lot of code mapping between the
> completely different legacy format (1.1.x, not re-scalable) and the
> key-group-oriented format (1.2.x onwards, re-scalable). It would greatly
> help the development of state and checkpointing features to drop that old
> code.
>
> Please let us know if you have concerns about that.
>
> Best,
> Stephan
>
>
Re: [POLL] Dropping savepoint format compatibility for 1.1.x in the
Flink 1.4.0 release
Posted by Ufuk Celebi <uc...@apache.org>.
On Fri, Jul 28, 2017 at 4:03 PM, Stephan Ewen <se...@apache.org> wrote:
> Seems like no one raised a concern so far about dropping the savepoint
> format compatibility for 1.1 in 1.4.
>
> Leaving this thread open for some more days, but from the sentiment, it
> seems like we should go ahead?
+1
Re: [POLL] Dropping savepoint format compatibility for 1.1.x in the
Flink 1.4.0 release
Posted by Stephan Ewen <se...@apache.org>.
@Greg - One benefit I can clearly see is the following:
If we keep that old 1.1-style state code, than we want to guarantee its
correctness in the face of the changes that have been made (consolidate
state code to be per-operator rather than per-task in the runtime as well)
and the changes that are WIP, for example for state evolution (like eager
state) or a better failover (don't reload state from DFS if the node did
not actually crash).
Guaranteeing that correctness is a lot of work. I think that not ensuring
the correctness as thoroughly would be worse that removing that code.
The CEP library had not really reached stable state in 1.2 anyways, it was
broken and reworked to introduce features like 'loops' or quantifiers. So
on that side, there would be no 1.2 compatibility anyways.
Do you see any concrete case were dropping 1.1 compatibility breaks any
setups? I personally know no user that is still on 1.1, most were very
eager about 1.2 due to rescalable state.
As someone that has also worked on the state code, I completely understand
Stefan's desire to simplify things there - it really slows down
developments in that component big time right now...
On Thu, Aug 17, 2017 at 1:34 PM, Stefan Richter <s.richter@data-artisans.com
> wrote:
> I think we are still doing changes for which this is relevant. Also I
> cannot really see a benefit in delaying this because the whole discussion
> will apply in exactly the same way to 1.5.
>
> > Am 17.08.2017 um 13:29 schrieb Greg Hogan <co...@greghogan.com>:
> >
> > There’s an argument for delaying this change to 1.5 since the feature
> freeze is two weeks away. There is little time to realize benefits from
> removing this code.
> >
> > "The reason for that is that there is a lot of code mapping between the
> completely different legacy format (1.1.x, not re-scalable) and the
> key-group-oriented format (1.2.x onwards, re-scalable). It would greatly
> help the development of state and checkpointing features to drop that old
> code.”
> >
> > Greg
> >
> >
> >> On Aug 17, 2017, at 5:36 AM, Stefan Richter <
> s.richter@data-artisans.com> wrote:
> >>
> >> One more comment about the consequences of this PR, as pointed out in
> the comments on Github: this will also break direct compatibility for the
> CEP library between Flink 1.2 and 1.4. There is still a way to migrate via
> Flink 1.3: Flink 1.1/2 -> savepoint -> Flink 1.3 -> savepoint -> Flink 1.4.
> >>
> >>> Am 16.08.2017 um 17:31 schrieb Stefan Richter <
> s.richter@data-artisans.com>:
> >>>
> >>> Hi,
> >>>
> >>> after there have been no objections since a long time, I took the next
> step and created a PR that implements this change in commit
> 95e44099784c9deaf2ca422b8dfc11c3d67d7f82 of https://github.com/apache/
> flink/pull/4550 <https://github.com/apache/flink/pull/4550> . Announcing
> this here as a last opportunity for further discussions. FYI, this will
> decrease the code base by almost 12K LOC.
> >>>
> >>> Best,
> >>> Stefan
> >>>
> >>>
> >>>> Am 02.08.2017 um 15:26 schrieb Kostas Kloudas <
> k.kloudas@data-artisans.com <ma...@data-artisans.com>>:
> >>>>
> >>>> +1
> >>>>
> >>>>> On Aug 2, 2017, at 3:16 PM, Till Rohrmann <trohrmann@apache.org
> <ma...@apache.org>> wrote:
> >>>>>
> >>>>> +1
> >>>>>
> >>>>> On Wed, Aug 2, 2017 at 9:12 AM, Stefan Richter <
> s.richter@data-artisans.com <ma...@data-artisans.com>>
> >>>>> wrote:
> >>>>>
> >>>>>> +1
> >>>>>>
> >>>>>> Am 28.07.2017 um 16:03 schrieb Stephan Ewen <sewen@apache.org
> <ma...@apache.org>>:
> >>>>>>
> >>>>>> Seems like no one raised a concern so far about dropping the
> savepoint
> >>>>>> format compatibility for 1.1 in 1.4.
> >>>>>>
> >>>>>> Leaving this thread open for some more days, but from the
> sentiment, it
> >>>>>> seems like we should go ahead?
> >>>>>>
> >>>>>> On Wed, Jul 12, 2017 at 4:43 PM, Stephan Ewen <sewen@apache.org
> <ma...@apache.org>> wrote:
> >>>>>>
> >>>>>>> Hi users!
> >>>>>>>
> >>>>>>> Flink currently maintains backwards compatibility for savepoint
> formats,
> >>>>>>> which means that savepoints taken with Flink version 1.1.x and
> 1.2.x can be
> >>>>>>> resumed in Flink 1.3.x
> >>>>>>>
> >>>>>>> We are discussing how many versions back to support. The
> proposition is
> >>>>>>> the following:
> >>>>>>>
> >>>>>>> * Suggestion: Flink 1.4.0 will be able to resume savepoints
> taken with
> >>>>>>> version 1.3.x and 1.2.x, but not savepoints from version 1.1.x and
> 1.0.x*
> >>>>>>>
> >>>>>>>
> >>>>>>> The reason for that is that there is a lot of code mapping between
> the
> >>>>>>> completely different legacy format (1.1.x, not re-scalable) and the
> >>>>>>> key-group-oriented format (1.2.x onwards, re-scalable). It would
> greatly
> >>>>>>> help the development of state and checkpointing features to drop
> that old
> >>>>>>> code.
> >>>>>>>
> >>>>>>> Please let us know if you have concerns about that.
> >>>>>>>
> >>>>>>> Best,
> >>>>>>> Stephan
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>
> >>>
> >>
> >
>
>
Re: [POLL] Dropping savepoint format compatibility for 1.1.x in the
Flink 1.4.0 release
Posted by Stefan Richter <s....@data-artisans.com>.
I think we are still doing changes for which this is relevant. Also I cannot really see a benefit in delaying this because the whole discussion will apply in exactly the same way to 1.5.
> Am 17.08.2017 um 13:29 schrieb Greg Hogan <co...@greghogan.com>:
>
> There’s an argument for delaying this change to 1.5 since the feature freeze is two weeks away. There is little time to realize benefits from removing this code.
>
> "The reason for that is that there is a lot of code mapping between the completely different legacy format (1.1.x, not re-scalable) and the key-group-oriented format (1.2.x onwards, re-scalable). It would greatly help the development of state and checkpointing features to drop that old code.”
>
> Greg
>
>
>> On Aug 17, 2017, at 5:36 AM, Stefan Richter <s....@data-artisans.com> wrote:
>>
>> One more comment about the consequences of this PR, as pointed out in the comments on Github: this will also break direct compatibility for the CEP library between Flink 1.2 and 1.4. There is still a way to migrate via Flink 1.3: Flink 1.1/2 -> savepoint -> Flink 1.3 -> savepoint -> Flink 1.4.
>>
>>> Am 16.08.2017 um 17:31 schrieb Stefan Richter <s....@data-artisans.com>:
>>>
>>> Hi,
>>>
>>> after there have been no objections since a long time, I took the next step and created a PR that implements this change in commit 95e44099784c9deaf2ca422b8dfc11c3d67d7f82 of https://github.com/apache/flink/pull/4550 <https://github.com/apache/flink/pull/4550> . Announcing this here as a last opportunity for further discussions. FYI, this will decrease the code base by almost 12K LOC.
>>>
>>> Best,
>>> Stefan
>>>
>>>
>>>> Am 02.08.2017 um 15:26 schrieb Kostas Kloudas <k.kloudas@data-artisans.com <ma...@data-artisans.com>>:
>>>>
>>>> +1
>>>>
>>>>> On Aug 2, 2017, at 3:16 PM, Till Rohrmann <trohrmann@apache.org <ma...@apache.org>> wrote:
>>>>>
>>>>> +1
>>>>>
>>>>> On Wed, Aug 2, 2017 at 9:12 AM, Stefan Richter <s.richter@data-artisans.com <ma...@data-artisans.com>>
>>>>> wrote:
>>>>>
>>>>>> +1
>>>>>>
>>>>>> Am 28.07.2017 um 16:03 schrieb Stephan Ewen <sewen@apache.org <ma...@apache.org>>:
>>>>>>
>>>>>> Seems like no one raised a concern so far about dropping the savepoint
>>>>>> format compatibility for 1.1 in 1.4.
>>>>>>
>>>>>> Leaving this thread open for some more days, but from the sentiment, it
>>>>>> seems like we should go ahead?
>>>>>>
>>>>>> On Wed, Jul 12, 2017 at 4:43 PM, Stephan Ewen <sewen@apache.org <ma...@apache.org>> wrote:
>>>>>>
>>>>>>> Hi users!
>>>>>>>
>>>>>>> Flink currently maintains backwards compatibility for savepoint formats,
>>>>>>> which means that savepoints taken with Flink version 1.1.x and 1.2.x can be
>>>>>>> resumed in Flink 1.3.x
>>>>>>>
>>>>>>> We are discussing how many versions back to support. The proposition is
>>>>>>> the following:
>>>>>>>
>>>>>>> * Suggestion: Flink 1.4.0 will be able to resume savepoints taken with
>>>>>>> version 1.3.x and 1.2.x, but not savepoints from version 1.1.x and 1.0.x*
>>>>>>>
>>>>>>>
>>>>>>> The reason for that is that there is a lot of code mapping between the
>>>>>>> completely different legacy format (1.1.x, not re-scalable) and the
>>>>>>> key-group-oriented format (1.2.x onwards, re-scalable). It would greatly
>>>>>>> help the development of state and checkpointing features to drop that old
>>>>>>> code.
>>>>>>>
>>>>>>> Please let us know if you have concerns about that.
>>>>>>>
>>>>>>> Best,
>>>>>>> Stephan
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>
>>>
>>
>
Re: [POLL] Dropping savepoint format compatibility for 1.1.x in the
Flink 1.4.0 release
Posted by Greg Hogan <co...@greghogan.com>.
There’s an argument for delaying this change to 1.5 since the feature freeze is two weeks away. There is little time to realize benefits from removing this code.
"The reason for that is that there is a lot of code mapping between the completely different legacy format (1.1.x, not re-scalable) and the key-group-oriented format (1.2.x onwards, re-scalable). It would greatly help the development of state and checkpointing features to drop that old code.”
Greg
> On Aug 17, 2017, at 5:36 AM, Stefan Richter <s....@data-artisans.com> wrote:
>
> One more comment about the consequences of this PR, as pointed out in the comments on Github: this will also break direct compatibility for the CEP library between Flink 1.2 and 1.4. There is still a way to migrate via Flink 1.3: Flink 1.1/2 -> savepoint -> Flink 1.3 -> savepoint -> Flink 1.4.
>
>> Am 16.08.2017 um 17:31 schrieb Stefan Richter <s....@data-artisans.com>:
>>
>> Hi,
>>
>> after there have been no objections since a long time, I took the next step and created a PR that implements this change in commit 95e44099784c9deaf2ca422b8dfc11c3d67d7f82 of https://github.com/apache/flink/pull/4550 <https://github.com/apache/flink/pull/4550> . Announcing this here as a last opportunity for further discussions. FYI, this will decrease the code base by almost 12K LOC.
>>
>> Best,
>> Stefan
>>
>>
>>> Am 02.08.2017 um 15:26 schrieb Kostas Kloudas <k.kloudas@data-artisans.com <ma...@data-artisans.com>>:
>>>
>>> +1
>>>
>>>> On Aug 2, 2017, at 3:16 PM, Till Rohrmann <trohrmann@apache.org <ma...@apache.org>> wrote:
>>>>
>>>> +1
>>>>
>>>> On Wed, Aug 2, 2017 at 9:12 AM, Stefan Richter <s.richter@data-artisans.com <ma...@data-artisans.com>>
>>>> wrote:
>>>>
>>>>> +1
>>>>>
>>>>> Am 28.07.2017 um 16:03 schrieb Stephan Ewen <sewen@apache.org <ma...@apache.org>>:
>>>>>
>>>>> Seems like no one raised a concern so far about dropping the savepoint
>>>>> format compatibility for 1.1 in 1.4.
>>>>>
>>>>> Leaving this thread open for some more days, but from the sentiment, it
>>>>> seems like we should go ahead?
>>>>>
>>>>> On Wed, Jul 12, 2017 at 4:43 PM, Stephan Ewen <sewen@apache.org <ma...@apache.org>> wrote:
>>>>>
>>>>>> Hi users!
>>>>>>
>>>>>> Flink currently maintains backwards compatibility for savepoint formats,
>>>>>> which means that savepoints taken with Flink version 1.1.x and 1.2.x can be
>>>>>> resumed in Flink 1.3.x
>>>>>>
>>>>>> We are discussing how many versions back to support. The proposition is
>>>>>> the following:
>>>>>>
>>>>>> * Suggestion: Flink 1.4.0 will be able to resume savepoints taken with
>>>>>> version 1.3.x and 1.2.x, but not savepoints from version 1.1.x and 1.0.x*
>>>>>>
>>>>>>
>>>>>> The reason for that is that there is a lot of code mapping between the
>>>>>> completely different legacy format (1.1.x, not re-scalable) and the
>>>>>> key-group-oriented format (1.2.x onwards, re-scalable). It would greatly
>>>>>> help the development of state and checkpointing features to drop that old
>>>>>> code.
>>>>>>
>>>>>> Please let us know if you have concerns about that.
>>>>>>
>>>>>> Best,
>>>>>> Stephan
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>
>>
>
Re: [POLL] Dropping savepoint format compatibility for 1.1.x in the
Flink 1.4.0 release
Posted by Stefan Richter <s....@data-artisans.com>.
One more comment about the consequences of this PR, as pointed out in the comments on Github: this will also break direct compatibility for the CEP library between Flink 1.2 and 1.4. There is still a way to migrate via Flink 1.3: Flink 1.1/2 -> savepoint -> Flink 1.3 -> savepoint -> Flink 1.4.
> Am 16.08.2017 um 17:31 schrieb Stefan Richter <s....@data-artisans.com>:
>
> Hi,
>
> after there have been no objections since a long time, I took the next step and created a PR that implements this change in commit 95e44099784c9deaf2ca422b8dfc11c3d67d7f82 of https://github.com/apache/flink/pull/4550 <https://github.com/apache/flink/pull/4550> . Announcing this here as a last opportunity for further discussions. FYI, this will decrease the code base by almost 12K LOC.
>
> Best,
> Stefan
>
>
>> Am 02.08.2017 um 15:26 schrieb Kostas Kloudas <k.kloudas@data-artisans.com <ma...@data-artisans.com>>:
>>
>> +1
>>
>>> On Aug 2, 2017, at 3:16 PM, Till Rohrmann <trohrmann@apache.org <ma...@apache.org>> wrote:
>>>
>>> +1
>>>
>>> On Wed, Aug 2, 2017 at 9:12 AM, Stefan Richter <s.richter@data-artisans.com <ma...@data-artisans.com>>
>>> wrote:
>>>
>>>> +1
>>>>
>>>> Am 28.07.2017 um 16:03 schrieb Stephan Ewen <sewen@apache.org <ma...@apache.org>>:
>>>>
>>>> Seems like no one raised a concern so far about dropping the savepoint
>>>> format compatibility for 1.1 in 1.4.
>>>>
>>>> Leaving this thread open for some more days, but from the sentiment, it
>>>> seems like we should go ahead?
>>>>
>>>> On Wed, Jul 12, 2017 at 4:43 PM, Stephan Ewen <sewen@apache.org <ma...@apache.org>> wrote:
>>>>
>>>>> Hi users!
>>>>>
>>>>> Flink currently maintains backwards compatibility for savepoint formats,
>>>>> which means that savepoints taken with Flink version 1.1.x and 1.2.x can be
>>>>> resumed in Flink 1.3.x
>>>>>
>>>>> We are discussing how many versions back to support. The proposition is
>>>>> the following:
>>>>>
>>>>> * Suggestion: Flink 1.4.0 will be able to resume savepoints taken with
>>>>> version 1.3.x and 1.2.x, but not savepoints from version 1.1.x and 1.0.x*
>>>>>
>>>>>
>>>>> The reason for that is that there is a lot of code mapping between the
>>>>> completely different legacy format (1.1.x, not re-scalable) and the
>>>>> key-group-oriented format (1.2.x onwards, re-scalable). It would greatly
>>>>> help the development of state and checkpointing features to drop that old
>>>>> code.
>>>>>
>>>>> Please let us know if you have concerns about that.
>>>>>
>>>>> Best,
>>>>> Stephan
>>>>>
>>>>>
>>>>
>>>>
>>
>
Re: [POLL] Dropping savepoint format compatibility for 1.1.x in the
Flink 1.4.0 release
Posted by Stefan Richter <s....@data-artisans.com>.
Hi,
after there have been no objections since a long time, I took the next step and created a PR that implements this change in commit 95e44099784c9deaf2ca422b8dfc11c3d67d7f82 of https://github.com/apache/flink/pull/4550 <https://github.com/apache/flink/pull/4550> . Announcing this here as a last opportunity for further discussions. FYI, this will decrease the code base by almost 12K LOC.
Best,
Stefan
> Am 02.08.2017 um 15:26 schrieb Kostas Kloudas <k....@data-artisans.com>:
>
> +1
>
>> On Aug 2, 2017, at 3:16 PM, Till Rohrmann <tr...@apache.org> wrote:
>>
>> +1
>>
>> On Wed, Aug 2, 2017 at 9:12 AM, Stefan Richter <s....@data-artisans.com>
>> wrote:
>>
>>> +1
>>>
>>> Am 28.07.2017 um 16:03 schrieb Stephan Ewen <se...@apache.org>:
>>>
>>> Seems like no one raised a concern so far about dropping the savepoint
>>> format compatibility for 1.1 in 1.4.
>>>
>>> Leaving this thread open for some more days, but from the sentiment, it
>>> seems like we should go ahead?
>>>
>>> On Wed, Jul 12, 2017 at 4:43 PM, Stephan Ewen <se...@apache.org> wrote:
>>>
>>>> Hi users!
>>>>
>>>> Flink currently maintains backwards compatibility for savepoint formats,
>>>> which means that savepoints taken with Flink version 1.1.x and 1.2.x can be
>>>> resumed in Flink 1.3.x
>>>>
>>>> We are discussing how many versions back to support. The proposition is
>>>> the following:
>>>>
>>>> * Suggestion: Flink 1.4.0 will be able to resume savepoints taken with
>>>> version 1.3.x and 1.2.x, but not savepoints from version 1.1.x and 1.0.x*
>>>>
>>>>
>>>> The reason for that is that there is a lot of code mapping between the
>>>> completely different legacy format (1.1.x, not re-scalable) and the
>>>> key-group-oriented format (1.2.x onwards, re-scalable). It would greatly
>>>> help the development of state and checkpointing features to drop that old
>>>> code.
>>>>
>>>> Please let us know if you have concerns about that.
>>>>
>>>> Best,
>>>> Stephan
>>>>
>>>>
>>>
>>>
>
Re: [POLL] Dropping savepoint format compatibility for 1.1.x in the
Flink 1.4.0 release
Posted by Kostas Kloudas <k....@data-artisans.com>.
+1
> On Aug 2, 2017, at 3:16 PM, Till Rohrmann <tr...@apache.org> wrote:
>
> +1
>
> On Wed, Aug 2, 2017 at 9:12 AM, Stefan Richter <s....@data-artisans.com>
> wrote:
>
>> +1
>>
>> Am 28.07.2017 um 16:03 schrieb Stephan Ewen <se...@apache.org>:
>>
>> Seems like no one raised a concern so far about dropping the savepoint
>> format compatibility for 1.1 in 1.4.
>>
>> Leaving this thread open for some more days, but from the sentiment, it
>> seems like we should go ahead?
>>
>> On Wed, Jul 12, 2017 at 4:43 PM, Stephan Ewen <se...@apache.org> wrote:
>>
>>> Hi users!
>>>
>>> Flink currently maintains backwards compatibility for savepoint formats,
>>> which means that savepoints taken with Flink version 1.1.x and 1.2.x can be
>>> resumed in Flink 1.3.x
>>>
>>> We are discussing how many versions back to support. The proposition is
>>> the following:
>>>
>>> * Suggestion: Flink 1.4.0 will be able to resume savepoints taken with
>>> version 1.3.x and 1.2.x, but not savepoints from version 1.1.x and 1.0.x*
>>>
>>>
>>> The reason for that is that there is a lot of code mapping between the
>>> completely different legacy format (1.1.x, not re-scalable) and the
>>> key-group-oriented format (1.2.x onwards, re-scalable). It would greatly
>>> help the development of state and checkpointing features to drop that old
>>> code.
>>>
>>> Please let us know if you have concerns about that.
>>>
>>> Best,
>>> Stephan
>>>
>>>
>>
>>
Re: [POLL] Dropping savepoint format compatibility for 1.1.x in the
Flink 1.4.0 release
Posted by Kostas Kloudas <k....@data-artisans.com>.
+1
> On Aug 2, 2017, at 3:16 PM, Till Rohrmann <tr...@apache.org> wrote:
>
> +1
>
> On Wed, Aug 2, 2017 at 9:12 AM, Stefan Richter <s....@data-artisans.com>
> wrote:
>
>> +1
>>
>> Am 28.07.2017 um 16:03 schrieb Stephan Ewen <se...@apache.org>:
>>
>> Seems like no one raised a concern so far about dropping the savepoint
>> format compatibility for 1.1 in 1.4.
>>
>> Leaving this thread open for some more days, but from the sentiment, it
>> seems like we should go ahead?
>>
>> On Wed, Jul 12, 2017 at 4:43 PM, Stephan Ewen <se...@apache.org> wrote:
>>
>>> Hi users!
>>>
>>> Flink currently maintains backwards compatibility for savepoint formats,
>>> which means that savepoints taken with Flink version 1.1.x and 1.2.x can be
>>> resumed in Flink 1.3.x
>>>
>>> We are discussing how many versions back to support. The proposition is
>>> the following:
>>>
>>> * Suggestion: Flink 1.4.0 will be able to resume savepoints taken with
>>> version 1.3.x and 1.2.x, but not savepoints from version 1.1.x and 1.0.x*
>>>
>>>
>>> The reason for that is that there is a lot of code mapping between the
>>> completely different legacy format (1.1.x, not re-scalable) and the
>>> key-group-oriented format (1.2.x onwards, re-scalable). It would greatly
>>> help the development of state and checkpointing features to drop that old
>>> code.
>>>
>>> Please let us know if you have concerns about that.
>>>
>>> Best,
>>> Stephan
>>>
>>>
>>
>>
Re: [POLL] Dropping savepoint format compatibility for 1.1.x in the
Flink 1.4.0 release
Posted by Till Rohrmann <tr...@apache.org>.
+1
On Wed, Aug 2, 2017 at 9:12 AM, Stefan Richter <s....@data-artisans.com>
wrote:
> +1
>
> Am 28.07.2017 um 16:03 schrieb Stephan Ewen <se...@apache.org>:
>
> Seems like no one raised a concern so far about dropping the savepoint
> format compatibility for 1.1 in 1.4.
>
> Leaving this thread open for some more days, but from the sentiment, it
> seems like we should go ahead?
>
> On Wed, Jul 12, 2017 at 4:43 PM, Stephan Ewen <se...@apache.org> wrote:
>
>> Hi users!
>>
>> Flink currently maintains backwards compatibility for savepoint formats,
>> which means that savepoints taken with Flink version 1.1.x and 1.2.x can be
>> resumed in Flink 1.3.x
>>
>> We are discussing how many versions back to support. The proposition is
>> the following:
>>
>> * Suggestion: Flink 1.4.0 will be able to resume savepoints taken with
>> version 1.3.x and 1.2.x, but not savepoints from version 1.1.x and 1.0.x*
>>
>>
>> The reason for that is that there is a lot of code mapping between the
>> completely different legacy format (1.1.x, not re-scalable) and the
>> key-group-oriented format (1.2.x onwards, re-scalable). It would greatly
>> help the development of state and checkpointing features to drop that old
>> code.
>>
>> Please let us know if you have concerns about that.
>>
>> Best,
>> Stephan
>>
>>
>
>
Re: [POLL] Dropping savepoint format compatibility for 1.1.x in the
Flink 1.4.0 release
Posted by Till Rohrmann <tr...@apache.org>.
+1
On Wed, Aug 2, 2017 at 9:12 AM, Stefan Richter <s....@data-artisans.com>
wrote:
> +1
>
> Am 28.07.2017 um 16:03 schrieb Stephan Ewen <se...@apache.org>:
>
> Seems like no one raised a concern so far about dropping the savepoint
> format compatibility for 1.1 in 1.4.
>
> Leaving this thread open for some more days, but from the sentiment, it
> seems like we should go ahead?
>
> On Wed, Jul 12, 2017 at 4:43 PM, Stephan Ewen <se...@apache.org> wrote:
>
>> Hi users!
>>
>> Flink currently maintains backwards compatibility for savepoint formats,
>> which means that savepoints taken with Flink version 1.1.x and 1.2.x can be
>> resumed in Flink 1.3.x
>>
>> We are discussing how many versions back to support. The proposition is
>> the following:
>>
>> * Suggestion: Flink 1.4.0 will be able to resume savepoints taken with
>> version 1.3.x and 1.2.x, but not savepoints from version 1.1.x and 1.0.x*
>>
>>
>> The reason for that is that there is a lot of code mapping between the
>> completely different legacy format (1.1.x, not re-scalable) and the
>> key-group-oriented format (1.2.x onwards, re-scalable). It would greatly
>> help the development of state and checkpointing features to drop that old
>> code.
>>
>> Please let us know if you have concerns about that.
>>
>> Best,
>> Stephan
>>
>>
>
>
Re: [POLL] Dropping savepoint format compatibility for 1.1.x in the
Flink 1.4.0 release
Posted by Stefan Richter <s....@data-artisans.com>.
+1
> Am 28.07.2017 um 16:03 schrieb Stephan Ewen <se...@apache.org>:
>
> Seems like no one raised a concern so far about dropping the savepoint format compatibility for 1.1 in 1.4.
>
> Leaving this thread open for some more days, but from the sentiment, it seems like we should go ahead?
>
> On Wed, Jul 12, 2017 at 4:43 PM, Stephan Ewen <sewen@apache.org <ma...@apache.org>> wrote:
> Hi users!
>
> Flink currently maintains backwards compatibility for savepoint formats, which means that savepoints taken with Flink version 1.1.x and 1.2.x can be resumed in Flink 1.3.x
>
> We are discussing how many versions back to support. The proposition is the following:
>
> Suggestion: Flink 1.4.0 will be able to resume savepoints taken with version 1.3.x and 1.2.x, but not savepoints from version 1.1.x and 1.0.x
>
>
> The reason for that is that there is a lot of code mapping between the completely different legacy format (1.1.x, not re-scalable) and the key-group-oriented format (1.2.x onwards, re-scalable). It would greatly help the development of state and checkpointing features to drop that old code.
>
> Please let us know if you have concerns about that.
>
> Best,
> Stephan
>
>
Re: [POLL] Dropping savepoint format compatibility for 1.1.x in the
Flink 1.4.0 release
Posted by Stefan Richter <s....@data-artisans.com>.
+1
> Am 28.07.2017 um 16:03 schrieb Stephan Ewen <se...@apache.org>:
>
> Seems like no one raised a concern so far about dropping the savepoint format compatibility for 1.1 in 1.4.
>
> Leaving this thread open for some more days, but from the sentiment, it seems like we should go ahead?
>
> On Wed, Jul 12, 2017 at 4:43 PM, Stephan Ewen <sewen@apache.org <ma...@apache.org>> wrote:
> Hi users!
>
> Flink currently maintains backwards compatibility for savepoint formats, which means that savepoints taken with Flink version 1.1.x and 1.2.x can be resumed in Flink 1.3.x
>
> We are discussing how many versions back to support. The proposition is the following:
>
> Suggestion: Flink 1.4.0 will be able to resume savepoints taken with version 1.3.x and 1.2.x, but not savepoints from version 1.1.x and 1.0.x
>
>
> The reason for that is that there is a lot of code mapping between the completely different legacy format (1.1.x, not re-scalable) and the key-group-oriented format (1.2.x onwards, re-scalable). It would greatly help the development of state and checkpointing features to drop that old code.
>
> Please let us know if you have concerns about that.
>
> Best,
> Stephan
>
>
Re: [POLL] Dropping savepoint format compatibility for 1.1.x in the
Flink 1.4.0 release
Posted by Ufuk Celebi <uc...@apache.org>.
On Fri, Jul 28, 2017 at 4:03 PM, Stephan Ewen <se...@apache.org> wrote:
> Seems like no one raised a concern so far about dropping the savepoint
> format compatibility for 1.1 in 1.4.
>
> Leaving this thread open for some more days, but from the sentiment, it
> seems like we should go ahead?
+1