You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by Joe Witt <jo...@apache.org> on 2023/01/09 20:53:00 UTC
[discuss] NiFi 1.20 and NiFi 2.0
Team,
As David mentioned in [1] following a successful NiFi 2.0 release goal
planning - we are now going to start moving the 'main' line to be the NiFi
2.0 line which will allow for the key work to take place. We will also
move niFi 1.x to its appropriate support line.
It is also the case that we have nearly 100 JIRAs on NiFi 1.20 and we have
work in there including security items so it is time to make a release.
The intent then is to initiate 1.20 and immediate after that change 'main'
to 2.0.
Going forward then all work on the 1.x line should be focused on
maintaining existing features, dependencies, and helping 1.x users migrate
to the 2.x line. Otherwise, new feature work will happen on 'main' as it
normally does and will come out in the NiFi 2.x release line.
Please flag key outstanding items as we narrow down the release candidate
for NiFi 1.20.
Thanks
Joe
[1] https://lists.apache.org/thread/qo4vvdw46235y7vy2crcd6l4m11wl7jz
Re: [discuss] NiFi 1.20 and NiFi 2.0
Posted by Phil H <gi...@gmail.com>.
Hi Joe,
Sorry if this is a dumb question, but is there some documentation around
what 2.0 might mean? My personal interest is in backwards compatibility of
existing custom processors.
Thanks,
Phil
On Tue, 10 Jan 2023 at 06:53, Joe Witt <jo...@apache.org> wrote:
> Team,
>
> As David mentioned in [1] following a successful NiFi 2.0 release goal
> planning - we are now going to start moving the 'main' line to be the NiFi
> 2.0 line which will allow for the key work to take place. We will also
> move niFi 1.x to its appropriate support line.
>
> It is also the case that we have nearly 100 JIRAs on NiFi 1.20 and we have
> work in there including security items so it is time to make a release.
> The intent then is to initiate 1.20 and immediate after that change 'main'
> to 2.0.
>
> Going forward then all work on the 1.x line should be focused on
> maintaining existing features, dependencies, and helping 1.x users migrate
> to the 2.x line. Otherwise, new feature work will happen on 'main' as it
> normally does and will come out in the NiFi 2.x release line.
>
> Please flag key outstanding items as we narrow down the release candidate
> for NiFi 1.20.
>
> Thanks
> Joe
>
> [1] https://lists.apache.org/thread/qo4vvdw46235y7vy2crcd6l4m11wl7jz
>
Re: [discuss] NiFi 1.20 and NiFi 2.0
Posted by Adam Taft <ad...@adamtaft.com>.
Hi David,
Thanks for the reply. I appreciate it, no further questions from me.
David wrote:
> For deployments that are not using deprecated components and
> features in 1.x...
That's going to be a hard one to sort out, as historically large NiFi
deployments are likely going to have at least some usage of deprecated
components in active production. And rolling out a "major" version change
will take quite a long time for these folks to get right.
If the 2.0 train moves ahead too quickly, and the 1.x releases stop and/or
are abandoned shortly afterwards, these types of "legacy" deployments are
going to suffer and will feel left behind. That's usually when communities
/ projects get forked. My timing/calendar questions are as much about
understanding the 1.x support lifecycle and keeping these slow movers alive
than anything. It would be bad if the rush to 2.0 comes at the expense of
existing deployments.
Again, thanks for the thorough reply.
/Adam
On Mon, Jan 9, 2023 at 6:52 PM David Handermann <ex...@apache.org>
wrote:
> Adam,
>
> Thanks for the reply.
>
> To clarify my statement about traction on 2.0 release goals, I stated it
> that way because I would like to see some measurable progress on the 2.0
> release goals before attempting to put forward any kind of timeline for
> release. I didn't mean to imply an increase in instability in the main
> branch, and that is not what I had in mind. On the contrary, the main
> branch should continue to be considered stable and current, and we should
> continue to apply the same level of rigor for all commits to the main
> branch. Changing major versions should not alter that approach.
>
> As outlined in the 2.0 Release Goals [1] the majority of the changes
> involve removal of deprecated components and features, as opposed to adding
> new and unstable capabilities. From that angle, a 2.0 major release should
> be stable, and the only concern should be that we are surgical in the
> removal process. That strategy is a large part of the reason that moving
> the main branch to 2.0.0-SNAPSHOT should not be seen as extremely
> disruptive. For deployments that are not using deprecated components and
> features in 1.x, upgrading to 2.x should be similar to upgrading between
> minor release versions.
>
> I appreciate the concern about potential disruption, and concern about
> stability. These concerns are important to keep in mind as we move forward.
> As long as we follow this approach, we should be able to maintain the level
> of stability that the community rightly expects.
>
> Regards,
> David Handermann
>
> [1]
> https://cwiki.apache.org/confluence/display/NIFI/NiFi+2.0+Release+Goals
>
>
>
> On Mon, Jan 9, 2023 at 6:08 PM Adam Taft <ad...@adamtaft.com> wrote:
>
> > I think this sentence is capturing some of my question ...
> >
> > David wrote:
> > > I think it would be helpful to see some traction on the 2.0 release
> goals
> > > before attempting to sketch out a potential timeline.
> >
> > It feels like what you're saying is that the "main" git branch is going
> to
> > become an alpha or beta for the 2.0.0 release, and that the newly
> proposed
> > "1.x" branch will be the stable branch. Without any existing traction on
> > the 2.0 release goals (as you've stated), it would start to feel that the
> > main branch doesn't maintain a predictable stability. Folks would have to
> > be looking at the 1.x branch for stable releases for an undefined period
> of
> > time.
> >
> > This is contrary to most philosophies as to what the "main" branch should
> > imply. Typically, the "alpha/beta" work for a major upcoming revision
> would
> > occur in a separate off-main branch until there is at least some fidelity
> > with the release goals. And then switching main from the 1.x to 2.x code
> > base would ideally happen as late as possible in the 2.0.0 release
> > candidate timeframe.
> >
> > It's splitting hairs, of course. Branches are just branches. But I do
> think
> > it's smart to keep the main branch tracking what is considered the
> > currently stable release, not a future beta. I can foresee that there
> will
> > be many 2.0.0 release candidates and late-adopter reluctance to jump onto
> > the 2.0 release until a few cycles of stability have been demonstrated.
> I'd
> > rather feel like we can recommend a 2.0 release straight out of the gate
> > rather than waiting for it to stabilize.
> >
> > No big deal here. Just trying to anticipate what to communicate to people
> > once main switches over. It sounds like the communication will be,
> "ignore
> > the main branch, and focus on the 1.x branch, if you want to be
> > conservative."
> >
> > /Adam
> >
> >
> >
> >
> > On Mon, Jan 9, 2023 at 3:24 PM David Handermann <
> > exceptionfactory@apache.org>
> > wrote:
> >
> > > Joe,
> > >
> > > Thanks for keeping things moving forward in terms of a 1.20 release and
> > 2.0
> > > branching plan. Releasing 1.20 and moving the main branch to
> > 2.0.0-SNAPSHOT
> > > aligns with the approved goals and provides a natural breakpoint for
> > > continued development on both branches.
> > >
> > > Adam,
> > >
> > > Thanks for raising the questions about timeline, I'm sure others have
> > > similar questions. I think it is probably a little too early to propose
> > > general timelines, but on the other hand, I think the historical pace
> of
> > > releases should be a good indication of continued release cadence.
> > >
> > > The 2.0 Release Goals did not include a timeline for the major release,
> > or
> > > subsequent minor releases, by design, but these are certainly questions
> > we
> > > should answer.
> > >
> > > We know that we will need at least one or 1.x releases to complete
> > > additional migration preparation work. With the scope of 2.0 Release
> > Goals
> > > purposefully limited, I would not expect extensive delays. We may need
> to
> > > have a longer release candidate period, or more incremental fix
> releases
> > > for the initial 2.0.0 release train, but I do not expect delaying a
> 2.0.0
> > > release for new features, as that is not part of the release goals.
> > >
> > > I think it would be helpful to see some traction on the 2.0 release
> goals
> > > before attempting to sketch out a potential timeline.
> > >
> > > Regards,
> > > David Handermann
> > >
> > > On Mon, Jan 9, 2023 at 3:50 PM Adam Taft <ad...@adamtaft.com> wrote:
> > >
> > > > Joe / team,
> > > >
> > > > Question on this. I think it would be helpful to understand the
> desired
> > > > timelines for the first 2.0.0 release. I know it's not strictly
> > > > predictable, but having a sense of what the timing looks like is
> > > important
> > > > to help understand the implications of a "maintenance only" 1.x line.
> > The
> > > > schedule would ideally come from the folks who are actively looking
> at
> > /
> > > > contributing to the 2.0 release. They probably have the best gauge as
> > to
> > > > "when" it might happen (under ideal conditions).
> > > >
> > > > One of the risks, of course, is if the 2.0 release stalls or delays.
> > > Having
> > > > an idea of how 1.x might evolve for the users who are not necessarily
> > > > early-adopters or those that need longer support tails. If 2.0 is
> > delayed
> > > > and 1.x looks unmaintained, there's a potential chance for the
> project
> > to
> > > > lose a bit of credibility. I know we don't anticipate this scenario,
> > but
> > > if
> > > > we had a plan for it, that would be reassuring.
> > > >
> > > > Maybe this was already addressed, I apologize if so. But if not, can
> we
> > > > throw some darts on the calendar to help understand the ideal rollout
> > of
> > > > 2.0 on a timeline? And are there any adjustments for the scenario
> > > described
> > > > above?
> > > >
> > > > Thanks in advance,
> > > >
> > > > /Adam
> > > >
> > > >
> > > > On Mon, Jan 9, 2023 at 1:53 PM Joe Witt <jo...@apache.org> wrote:
> > > >
> > > > > Team,
> > > > >
> > > > > As David mentioned in [1] following a successful NiFi 2.0 release
> > goal
> > > > > planning - we are now going to start moving the 'main' line to be
> the
> > > > NiFi
> > > > > 2.0 line which will allow for the key work to take place. We will
> > also
> > > > > move niFi 1.x to its appropriate support line.
> > > > >
> > > > > It is also the case that we have nearly 100 JIRAs on NiFi 1.20 and
> we
> > > > have
> > > > > work in there including security items so it is time to make a
> > release.
> > > > > The intent then is to initiate 1.20 and immediate after that change
> > > > 'main'
> > > > > to 2.0.
> > > > >
> > > > > Going forward then all work on the 1.x line should be focused on
> > > > > maintaining existing features, dependencies, and helping 1.x users
> > > > migrate
> > > > > to the 2.x line. Otherwise, new feature work will happen on 'main'
> > as
> > > it
> > > > > normally does and will come out in the NiFi 2.x release line.
> > > > >
> > > > > Please flag key outstanding items as we narrow down the release
> > > candidate
> > > > > for NiFi 1.20.
> > > > >
> > > > > Thanks
> > > > > Joe
> > > > >
> > > > > [1]
> https://lists.apache.org/thread/qo4vvdw46235y7vy2crcd6l4m11wl7jz
> > > > >
> > > >
> > >
> >
>
Re: [discuss] NiFi 1.20 and NiFi 2.0
Posted by David Handermann <ex...@apache.org>.
Adam,
Thanks for the reply.
To clarify my statement about traction on 2.0 release goals, I stated it
that way because I would like to see some measurable progress on the 2.0
release goals before attempting to put forward any kind of timeline for
release. I didn't mean to imply an increase in instability in the main
branch, and that is not what I had in mind. On the contrary, the main
branch should continue to be considered stable and current, and we should
continue to apply the same level of rigor for all commits to the main
branch. Changing major versions should not alter that approach.
As outlined in the 2.0 Release Goals [1] the majority of the changes
involve removal of deprecated components and features, as opposed to adding
new and unstable capabilities. From that angle, a 2.0 major release should
be stable, and the only concern should be that we are surgical in the
removal process. That strategy is a large part of the reason that moving
the main branch to 2.0.0-SNAPSHOT should not be seen as extremely
disruptive. For deployments that are not using deprecated components and
features in 1.x, upgrading to 2.x should be similar to upgrading between
minor release versions.
I appreciate the concern about potential disruption, and concern about
stability. These concerns are important to keep in mind as we move forward.
As long as we follow this approach, we should be able to maintain the level
of stability that the community rightly expects.
Regards,
David Handermann
[1] https://cwiki.apache.org/confluence/display/NIFI/NiFi+2.0+Release+Goals
On Mon, Jan 9, 2023 at 6:08 PM Adam Taft <ad...@adamtaft.com> wrote:
> I think this sentence is capturing some of my question ...
>
> David wrote:
> > I think it would be helpful to see some traction on the 2.0 release goals
> > before attempting to sketch out a potential timeline.
>
> It feels like what you're saying is that the "main" git branch is going to
> become an alpha or beta for the 2.0.0 release, and that the newly proposed
> "1.x" branch will be the stable branch. Without any existing traction on
> the 2.0 release goals (as you've stated), it would start to feel that the
> main branch doesn't maintain a predictable stability. Folks would have to
> be looking at the 1.x branch for stable releases for an undefined period of
> time.
>
> This is contrary to most philosophies as to what the "main" branch should
> imply. Typically, the "alpha/beta" work for a major upcoming revision would
> occur in a separate off-main branch until there is at least some fidelity
> with the release goals. And then switching main from the 1.x to 2.x code
> base would ideally happen as late as possible in the 2.0.0 release
> candidate timeframe.
>
> It's splitting hairs, of course. Branches are just branches. But I do think
> it's smart to keep the main branch tracking what is considered the
> currently stable release, not a future beta. I can foresee that there will
> be many 2.0.0 release candidates and late-adopter reluctance to jump onto
> the 2.0 release until a few cycles of stability have been demonstrated. I'd
> rather feel like we can recommend a 2.0 release straight out of the gate
> rather than waiting for it to stabilize.
>
> No big deal here. Just trying to anticipate what to communicate to people
> once main switches over. It sounds like the communication will be, "ignore
> the main branch, and focus on the 1.x branch, if you want to be
> conservative."
>
> /Adam
>
>
>
>
> On Mon, Jan 9, 2023 at 3:24 PM David Handermann <
> exceptionfactory@apache.org>
> wrote:
>
> > Joe,
> >
> > Thanks for keeping things moving forward in terms of a 1.20 release and
> 2.0
> > branching plan. Releasing 1.20 and moving the main branch to
> 2.0.0-SNAPSHOT
> > aligns with the approved goals and provides a natural breakpoint for
> > continued development on both branches.
> >
> > Adam,
> >
> > Thanks for raising the questions about timeline, I'm sure others have
> > similar questions. I think it is probably a little too early to propose
> > general timelines, but on the other hand, I think the historical pace of
> > releases should be a good indication of continued release cadence.
> >
> > The 2.0 Release Goals did not include a timeline for the major release,
> or
> > subsequent minor releases, by design, but these are certainly questions
> we
> > should answer.
> >
> > We know that we will need at least one or 1.x releases to complete
> > additional migration preparation work. With the scope of 2.0 Release
> Goals
> > purposefully limited, I would not expect extensive delays. We may need to
> > have a longer release candidate period, or more incremental fix releases
> > for the initial 2.0.0 release train, but I do not expect delaying a 2.0.0
> > release for new features, as that is not part of the release goals.
> >
> > I think it would be helpful to see some traction on the 2.0 release goals
> > before attempting to sketch out a potential timeline.
> >
> > Regards,
> > David Handermann
> >
> > On Mon, Jan 9, 2023 at 3:50 PM Adam Taft <ad...@adamtaft.com> wrote:
> >
> > > Joe / team,
> > >
> > > Question on this. I think it would be helpful to understand the desired
> > > timelines for the first 2.0.0 release. I know it's not strictly
> > > predictable, but having a sense of what the timing looks like is
> > important
> > > to help understand the implications of a "maintenance only" 1.x line.
> The
> > > schedule would ideally come from the folks who are actively looking at
> /
> > > contributing to the 2.0 release. They probably have the best gauge as
> to
> > > "when" it might happen (under ideal conditions).
> > >
> > > One of the risks, of course, is if the 2.0 release stalls or delays.
> > Having
> > > an idea of how 1.x might evolve for the users who are not necessarily
> > > early-adopters or those that need longer support tails. If 2.0 is
> delayed
> > > and 1.x looks unmaintained, there's a potential chance for the project
> to
> > > lose a bit of credibility. I know we don't anticipate this scenario,
> but
> > if
> > > we had a plan for it, that would be reassuring.
> > >
> > > Maybe this was already addressed, I apologize if so. But if not, can we
> > > throw some darts on the calendar to help understand the ideal rollout
> of
> > > 2.0 on a timeline? And are there any adjustments for the scenario
> > described
> > > above?
> > >
> > > Thanks in advance,
> > >
> > > /Adam
> > >
> > >
> > > On Mon, Jan 9, 2023 at 1:53 PM Joe Witt <jo...@apache.org> wrote:
> > >
> > > > Team,
> > > >
> > > > As David mentioned in [1] following a successful NiFi 2.0 release
> goal
> > > > planning - we are now going to start moving the 'main' line to be the
> > > NiFi
> > > > 2.0 line which will allow for the key work to take place. We will
> also
> > > > move niFi 1.x to its appropriate support line.
> > > >
> > > > It is also the case that we have nearly 100 JIRAs on NiFi 1.20 and we
> > > have
> > > > work in there including security items so it is time to make a
> release.
> > > > The intent then is to initiate 1.20 and immediate after that change
> > > 'main'
> > > > to 2.0.
> > > >
> > > > Going forward then all work on the 1.x line should be focused on
> > > > maintaining existing features, dependencies, and helping 1.x users
> > > migrate
> > > > to the 2.x line. Otherwise, new feature work will happen on 'main'
> as
> > it
> > > > normally does and will come out in the NiFi 2.x release line.
> > > >
> > > > Please flag key outstanding items as we narrow down the release
> > candidate
> > > > for NiFi 1.20.
> > > >
> > > > Thanks
> > > > Joe
> > > >
> > > > [1] https://lists.apache.org/thread/qo4vvdw46235y7vy2crcd6l4m11wl7jz
> > > >
> > >
> >
>
Re: [discuss] NiFi 1.20 and NiFi 2.0
Posted by Adam Taft <ad...@adamtaft.com>.
I think this sentence is capturing some of my question ...
David wrote:
> I think it would be helpful to see some traction on the 2.0 release goals
> before attempting to sketch out a potential timeline.
It feels like what you're saying is that the "main" git branch is going to
become an alpha or beta for the 2.0.0 release, and that the newly proposed
"1.x" branch will be the stable branch. Without any existing traction on
the 2.0 release goals (as you've stated), it would start to feel that the
main branch doesn't maintain a predictable stability. Folks would have to
be looking at the 1.x branch for stable releases for an undefined period of
time.
This is contrary to most philosophies as to what the "main" branch should
imply. Typically, the "alpha/beta" work for a major upcoming revision would
occur in a separate off-main branch until there is at least some fidelity
with the release goals. And then switching main from the 1.x to 2.x code
base would ideally happen as late as possible in the 2.0.0 release
candidate timeframe.
It's splitting hairs, of course. Branches are just branches. But I do think
it's smart to keep the main branch tracking what is considered the
currently stable release, not a future beta. I can foresee that there will
be many 2.0.0 release candidates and late-adopter reluctance to jump onto
the 2.0 release until a few cycles of stability have been demonstrated. I'd
rather feel like we can recommend a 2.0 release straight out of the gate
rather than waiting for it to stabilize.
No big deal here. Just trying to anticipate what to communicate to people
once main switches over. It sounds like the communication will be, "ignore
the main branch, and focus on the 1.x branch, if you want to be
conservative."
/Adam
On Mon, Jan 9, 2023 at 3:24 PM David Handermann <ex...@apache.org>
wrote:
> Joe,
>
> Thanks for keeping things moving forward in terms of a 1.20 release and 2.0
> branching plan. Releasing 1.20 and moving the main branch to 2.0.0-SNAPSHOT
> aligns with the approved goals and provides a natural breakpoint for
> continued development on both branches.
>
> Adam,
>
> Thanks for raising the questions about timeline, I'm sure others have
> similar questions. I think it is probably a little too early to propose
> general timelines, but on the other hand, I think the historical pace of
> releases should be a good indication of continued release cadence.
>
> The 2.0 Release Goals did not include a timeline for the major release, or
> subsequent minor releases, by design, but these are certainly questions we
> should answer.
>
> We know that we will need at least one or 1.x releases to complete
> additional migration preparation work. With the scope of 2.0 Release Goals
> purposefully limited, I would not expect extensive delays. We may need to
> have a longer release candidate period, or more incremental fix releases
> for the initial 2.0.0 release train, but I do not expect delaying a 2.0.0
> release for new features, as that is not part of the release goals.
>
> I think it would be helpful to see some traction on the 2.0 release goals
> before attempting to sketch out a potential timeline.
>
> Regards,
> David Handermann
>
> On Mon, Jan 9, 2023 at 3:50 PM Adam Taft <ad...@adamtaft.com> wrote:
>
> > Joe / team,
> >
> > Question on this. I think it would be helpful to understand the desired
> > timelines for the first 2.0.0 release. I know it's not strictly
> > predictable, but having a sense of what the timing looks like is
> important
> > to help understand the implications of a "maintenance only" 1.x line. The
> > schedule would ideally come from the folks who are actively looking at /
> > contributing to the 2.0 release. They probably have the best gauge as to
> > "when" it might happen (under ideal conditions).
> >
> > One of the risks, of course, is if the 2.0 release stalls or delays.
> Having
> > an idea of how 1.x might evolve for the users who are not necessarily
> > early-adopters or those that need longer support tails. If 2.0 is delayed
> > and 1.x looks unmaintained, there's a potential chance for the project to
> > lose a bit of credibility. I know we don't anticipate this scenario, but
> if
> > we had a plan for it, that would be reassuring.
> >
> > Maybe this was already addressed, I apologize if so. But if not, can we
> > throw some darts on the calendar to help understand the ideal rollout of
> > 2.0 on a timeline? And are there any adjustments for the scenario
> described
> > above?
> >
> > Thanks in advance,
> >
> > /Adam
> >
> >
> > On Mon, Jan 9, 2023 at 1:53 PM Joe Witt <jo...@apache.org> wrote:
> >
> > > Team,
> > >
> > > As David mentioned in [1] following a successful NiFi 2.0 release goal
> > > planning - we are now going to start moving the 'main' line to be the
> > NiFi
> > > 2.0 line which will allow for the key work to take place. We will also
> > > move niFi 1.x to its appropriate support line.
> > >
> > > It is also the case that we have nearly 100 JIRAs on NiFi 1.20 and we
> > have
> > > work in there including security items so it is time to make a release.
> > > The intent then is to initiate 1.20 and immediate after that change
> > 'main'
> > > to 2.0.
> > >
> > > Going forward then all work on the 1.x line should be focused on
> > > maintaining existing features, dependencies, and helping 1.x users
> > migrate
> > > to the 2.x line. Otherwise, new feature work will happen on 'main' as
> it
> > > normally does and will come out in the NiFi 2.x release line.
> > >
> > > Please flag key outstanding items as we narrow down the release
> candidate
> > > for NiFi 1.20.
> > >
> > > Thanks
> > > Joe
> > >
> > > [1] https://lists.apache.org/thread/qo4vvdw46235y7vy2crcd6l4m11wl7jz
> > >
> >
>
Re: [discuss] NiFi 1.20 and NiFi 2.0
Posted by David Handermann <ex...@apache.org>.
Joe,
Thanks for keeping things moving forward in terms of a 1.20 release and 2.0
branching plan. Releasing 1.20 and moving the main branch to 2.0.0-SNAPSHOT
aligns with the approved goals and provides a natural breakpoint for
continued development on both branches.
Adam,
Thanks for raising the questions about timeline, I'm sure others have
similar questions. I think it is probably a little too early to propose
general timelines, but on the other hand, I think the historical pace of
releases should be a good indication of continued release cadence.
The 2.0 Release Goals did not include a timeline for the major release, or
subsequent minor releases, by design, but these are certainly questions we
should answer.
We know that we will need at least one or 1.x releases to complete
additional migration preparation work. With the scope of 2.0 Release Goals
purposefully limited, I would not expect extensive delays. We may need to
have a longer release candidate period, or more incremental fix releases
for the initial 2.0.0 release train, but I do not expect delaying a 2.0.0
release for new features, as that is not part of the release goals.
I think it would be helpful to see some traction on the 2.0 release goals
before attempting to sketch out a potential timeline.
Regards,
David Handermann
On Mon, Jan 9, 2023 at 3:50 PM Adam Taft <ad...@adamtaft.com> wrote:
> Joe / team,
>
> Question on this. I think it would be helpful to understand the desired
> timelines for the first 2.0.0 release. I know it's not strictly
> predictable, but having a sense of what the timing looks like is important
> to help understand the implications of a "maintenance only" 1.x line. The
> schedule would ideally come from the folks who are actively looking at /
> contributing to the 2.0 release. They probably have the best gauge as to
> "when" it might happen (under ideal conditions).
>
> One of the risks, of course, is if the 2.0 release stalls or delays. Having
> an idea of how 1.x might evolve for the users who are not necessarily
> early-adopters or those that need longer support tails. If 2.0 is delayed
> and 1.x looks unmaintained, there's a potential chance for the project to
> lose a bit of credibility. I know we don't anticipate this scenario, but if
> we had a plan for it, that would be reassuring.
>
> Maybe this was already addressed, I apologize if so. But if not, can we
> throw some darts on the calendar to help understand the ideal rollout of
> 2.0 on a timeline? And are there any adjustments for the scenario described
> above?
>
> Thanks in advance,
>
> /Adam
>
>
> On Mon, Jan 9, 2023 at 1:53 PM Joe Witt <jo...@apache.org> wrote:
>
> > Team,
> >
> > As David mentioned in [1] following a successful NiFi 2.0 release goal
> > planning - we are now going to start moving the 'main' line to be the
> NiFi
> > 2.0 line which will allow for the key work to take place. We will also
> > move niFi 1.x to its appropriate support line.
> >
> > It is also the case that we have nearly 100 JIRAs on NiFi 1.20 and we
> have
> > work in there including security items so it is time to make a release.
> > The intent then is to initiate 1.20 and immediate after that change
> 'main'
> > to 2.0.
> >
> > Going forward then all work on the 1.x line should be focused on
> > maintaining existing features, dependencies, and helping 1.x users
> migrate
> > to the 2.x line. Otherwise, new feature work will happen on 'main' as it
> > normally does and will come out in the NiFi 2.x release line.
> >
> > Please flag key outstanding items as we narrow down the release candidate
> > for NiFi 1.20.
> >
> > Thanks
> > Joe
> >
> > [1] https://lists.apache.org/thread/qo4vvdw46235y7vy2crcd6l4m11wl7jz
> >
>
Re: [discuss] NiFi 1.20 and NiFi 2.0
Posted by Ryan Hendrickson <ry...@gmail.com>.
I second some calendar guessing. There's been releases every 1-2 months
for a while now. I'd be curious what a guess at a gap would be between
1.20.0 and a GA of 2.0.0.
Thanks,
Ryan
On Mon, Jan 9, 2023 at 4:50 PM Adam Taft <ad...@adamtaft.com> wrote:
> Joe / team,
>
> Question on this. I think it would be helpful to understand the desired
> timelines for the first 2.0.0 release. I know it's not strictly
> predictable, but having a sense of what the timing looks like is important
> to help understand the implications of a "maintenance only" 1.x line. The
> schedule would ideally come from the folks who are actively looking at /
> contributing to the 2.0 release. They probably have the best gauge as to
> "when" it might happen (under ideal conditions).
>
> One of the risks, of course, is if the 2.0 release stalls or delays. Having
> an idea of how 1.x might evolve for the users who are not necessarily
> early-adopters or those that need longer support tails. If 2.0 is delayed
> and 1.x looks unmaintained, there's a potential chance for the project to
> lose a bit of credibility. I know we don't anticipate this scenario, but if
> we had a plan for it, that would be reassuring.
>
> Maybe this was already addressed, I apologize if so. But if not, can we
> throw some darts on the calendar to help understand the ideal rollout of
> 2.0 on a timeline? And are there any adjustments for the scenario described
> above?
>
> Thanks in advance,
>
> /Adam
>
>
> On Mon, Jan 9, 2023 at 1:53 PM Joe Witt <jo...@apache.org> wrote:
>
> > Team,
> >
> > As David mentioned in [1] following a successful NiFi 2.0 release goal
> > planning - we are now going to start moving the 'main' line to be the
> NiFi
> > 2.0 line which will allow for the key work to take place. We will also
> > move niFi 1.x to its appropriate support line.
> >
> > It is also the case that we have nearly 100 JIRAs on NiFi 1.20 and we
> have
> > work in there including security items so it is time to make a release.
> > The intent then is to initiate 1.20 and immediate after that change
> 'main'
> > to 2.0.
> >
> > Going forward then all work on the 1.x line should be focused on
> > maintaining existing features, dependencies, and helping 1.x users
> migrate
> > to the 2.x line. Otherwise, new feature work will happen on 'main' as it
> > normally does and will come out in the NiFi 2.x release line.
> >
> > Please flag key outstanding items as we narrow down the release candidate
> > for NiFi 1.20.
> >
> > Thanks
> > Joe
> >
> > [1] https://lists.apache.org/thread/qo4vvdw46235y7vy2crcd6l4m11wl7jz
> >
>
Re: [discuss] NiFi 1.20 and NiFi 2.0
Posted by Adam Taft <ad...@adamtaft.com>.
Joe / team,
Question on this. I think it would be helpful to understand the desired
timelines for the first 2.0.0 release. I know it's not strictly
predictable, but having a sense of what the timing looks like is important
to help understand the implications of a "maintenance only" 1.x line. The
schedule would ideally come from the folks who are actively looking at /
contributing to the 2.0 release. They probably have the best gauge as to
"when" it might happen (under ideal conditions).
One of the risks, of course, is if the 2.0 release stalls or delays. Having
an idea of how 1.x might evolve for the users who are not necessarily
early-adopters or those that need longer support tails. If 2.0 is delayed
and 1.x looks unmaintained, there's a potential chance for the project to
lose a bit of credibility. I know we don't anticipate this scenario, but if
we had a plan for it, that would be reassuring.
Maybe this was already addressed, I apologize if so. But if not, can we
throw some darts on the calendar to help understand the ideal rollout of
2.0 on a timeline? And are there any adjustments for the scenario described
above?
Thanks in advance,
/Adam
On Mon, Jan 9, 2023 at 1:53 PM Joe Witt <jo...@apache.org> wrote:
> Team,
>
> As David mentioned in [1] following a successful NiFi 2.0 release goal
> planning - we are now going to start moving the 'main' line to be the NiFi
> 2.0 line which will allow for the key work to take place. We will also
> move niFi 1.x to its appropriate support line.
>
> It is also the case that we have nearly 100 JIRAs on NiFi 1.20 and we have
> work in there including security items so it is time to make a release.
> The intent then is to initiate 1.20 and immediate after that change 'main'
> to 2.0.
>
> Going forward then all work on the 1.x line should be focused on
> maintaining existing features, dependencies, and helping 1.x users migrate
> to the 2.x line. Otherwise, new feature work will happen on 'main' as it
> normally does and will come out in the NiFi 2.x release line.
>
> Please flag key outstanding items as we narrow down the release candidate
> for NiFi 1.20.
>
> Thanks
> Joe
>
> [1] https://lists.apache.org/thread/qo4vvdw46235y7vy2crcd6l4m11wl7jz
>
Re: [discuss] NiFi 1.20 and NiFi 2.0
Posted by Otto Fowler <ot...@gmail.com>.
Super, is that written down anywhere we can point people to?
From: Bryan Bende <bb...@gmail.com> <bb...@gmail.com>
Reply: dev@nifi.apache.org <de...@nifi.apache.org> <de...@nifi.apache.org>
Date: January 10, 2023 at 09:32:51
To: dev@nifi.apache.org <de...@nifi.apache.org> <de...@nifi.apache.org>
Subject: Re: [discuss] NiFi 1.20 and NiFi 2.0
The plan as I understand it is not to diverge and create separate feature
development on the 1.x line, so I would expect all PRs to continue to be
submitted only to main. We would release 1.x as needed with major bug fixes
or critical security updates, and these would be cherry-picked and/or
backported as necessary, mostly without the need for PRs, the same as we
would do if we were bringing fixes from main (1.20.0-SNAPSHOT) back to a
maintenance line like (1.19.x). For precedent, we followed this same
approach going from the 0.x line to 1.0.0 and there wasn't any major issue.
On Tue, Jan 10, 2023 at 7:07 AM Otto Fowler <ot...@gmail.com>
wrote:
> It was also mentioned in another thread that we need to have agreement on
> our explicit strategy and support for 1.x going forward, did that happen?
>
> From: Otto Fowler <ot...@gmail.com> <ot...@gmail.com>
> Reply: Otto Fowler <ot...@gmail.com> <ot...@gmail.com>
> Date: January 10, 2023 at 07:02:34
> To: dev@nifi.apache.org <de...@nifi.apache.org> <de...@nifi.apache.org>
> Subject: Re: [discuss] NiFi 1.20 and NiFi 2.0
>
> There needs to be an update to the contributing guide as to how to submit
> PRs to 1.x or 2.x etc.
>
> From: Joe Witt <jo...@apache.org> <jo...@apache.org>
> Reply: dev@nifi.apache.org <de...@nifi.apache.org> <de...@nifi.apache.org>
> Date: January 9, 2023 at 15:53:16
> To: dev@nifi.apache.org <de...@nifi.apache.org> <de...@nifi.apache.org>
> Subject: [discuss] NiFi 1.20 and NiFi 2.0
>
> Team,
>
> As David mentioned in [1] following a successful NiFi 2.0 release goal
> planning - we are now going to start moving the 'main' line to be the
NiFi
> 2.0 line which will allow for the key work to take place. We will also
> move niFi 1.x to its appropriate support line.
>
> It is also the case that we have nearly 100 JIRAs on NiFi 1.20 and we
have
> work in there including security items so it is time to make a release.
> The intent then is to initiate 1.20 and immediate after that change
'main'
> to 2.0.
>
> Going forward then all work on the 1.x line should be focused on
> maintaining existing features, dependencies, and helping 1.x users
migrate
> to the 2.x line. Otherwise, new feature work will happen on 'main' as it
> normally does and will come out in the NiFi 2.x release line.
>
> Please flag key outstanding items as we narrow down the release candidate
> for NiFi 1.20.
>
> Thanks
> Joe
>
> [1] https://lists.apache.org/thread/qo4vvdw46235y7vy2crcd6l4m11wl7jz
>
Re: [discuss] NiFi 1.20 and NiFi 2.0
Posted by Kevin Doran <kd...@gmail.com>.
I agree main needs to target 2.x, that is where most effort should be focused, and development there can be cherry picked / backported to a 1.x branch. The inverse approach creates too much of a moving target for 2.x.
Joe, I do not see anything else needed for cutting 1.20.
________________________________
From: Joe Witt <jo...@gmail.com>
Sent: Wednesday, January 11, 2023 7:05:59 PM
To: dev@nifi.apache.org <de...@nifi.apache.org>
Subject: Re: [discuss] NiFi 1.20 and NiFi 2.0
Hello
We dont have a release schedule and never really have. What we do have is
7+ years of track record which shows who and what we do. We have done a
major release change before. We release every couple of months or so.
This is very manageable and the stated goals are scoped as such.
We have declared what our goals are for 2.0 and now it is time to simply
make it happen. We will keep 1.x moving along in the meantime but one of
these is going to get the ‘most’ energy as that is ultimately how all this
works.
Whatever is our ‘main’ line is where all PRs go by default and where people
work by default. The burden of maintaining these lines is very real and
will provide plenty of motivation to move 2.0 along rapidly. The true
notion of business as usual is where PRs land and who does the work to
review and merge them.
The commentary around production usage worries suggests ideas or views that
differ from the discussed, documented and now voted upon release goals.
Someone starting on 2.0 should enjoy stability similar to a NiFi 1.20
release. Existing users will be provided tooling/automation/docs to move
from 1.x to 2.x.
Milestones may serve as a valuable tool
for us. We can use them as we vet out migration tooling we put on the 1.x
line.
Do we need anything else for 1.20?
Thanks
On Wed, Jan 11, 2023 at 3:42 PM Adam Taft <ad...@adamtaft.com> wrote:
> This is really insightful and spot on ...
>
> Kevin wrote:
> > Good migration tooling will take a while to develop and test, and the
> core
> > contributors to that effort may not have sufficient variety of flows to
> > evaluate when the migration tools are "done" for the majority of the
> > community to have success upgrading to 2.x. A milestone release would
> allow
> > us get more feedback on migration over a longer period than the vote
> window
> > of an RC candidate.
>
> It's exactly this case, that an early 2.0 release might not have had time
> to fully work its way through existing production deployments, that's
> concerning. The pace and voting of an "RC" is much too short to get any
> quality feedback from users in the field.
>
> I think it's really smart to consider the "Milestone" release approach
> here. We release 2.0.0-M1, 2.0.0-M2, ... waiting an adequate amount of time
> for feedback. We can put these milestones on a calendar, as needed, so that
> feedback is required some 'x' number of weeks/months after each milestone.
>
> And to this end, I'd personally rather see us keep the 'main' branch
> current with the 1.x line _until_ we're ready and are satisfied with the
> end goals of the 2.0 release objectives. When the milestone releases have
> been completed and there's a comfort level with the 2.x line, it's at the
> point we'd isolate the 1.x line into its own branch and switch main over to
> the 2.x line.
>
> This is an attractive way of:
> a) continuing business-as-usual with the 1.x line
> b) making headway on the 2.x release milestones
> c) giving adequate time for feedback against the 2.0 milestones coming from
> the field
>
> I don't mind the known-unknowns. But it's really the unknown-unknowns that
> are going to drive a delay in the 2.0 release. I think it's smart to be
> able to get some of the unknowns ironed out before we finalize the 2.0
> release ceremony. The milestone approach really helps with that.
>
> /Adam
>
> On Wed, Jan 11, 2023 at 11:11 AM Kevin Doran <kd...@apache.org> wrote:
>
> > Sorry, Joe, I was not clear, and to be honest the two thoughts are
> somewhat
> > unrelated in my mind too :)
> >
> > I agree that good migration tooling is key. Otherwise, we risk users
> > staying on 1.x or creating a schism of 1.x and 2.x users.
> >
> > Good migration tooling will take a while to develop and test, and the
> core
> > contributors to that effort may not have sufficient variety of flows to
> > evaluate when the migration tools are "done" for the majority of the
> > community to have success upgrading to 2.x. A milestone release would
> allow
> > us get more feedback on migration over a longer period than the vote
> window
> > of an RC candidate.
> >
> > Perhaps we could continue to release from the 1.x line (including minor
> > releases with some features) until we are ready to drop the "milestone"
> > qualifier from 2.0.0, and only then put 1.x into maintenance-only status.
> > It would be the same proposal to move main to target 2.0.0-M1, but relax
> > restrictions for what can land on the 1.x branch and be open to a 1.21,
> > 1.22, etc. if 2.0.0 work takes longer than anticipated. For example,
> maybe
> > we would be open to landing new/backported processors on the 1.x branch,
> > but not core framework features or API changes.
> >
> > This might not be necessary, but I think it is fair that saying "no new
> > features on 1.x" and also "no new features in 2.0.0" puts the project in
> a
> > rough place if 2.0.0 takes longer than a few months, so if we go that
> > route, we need to commit to a quick release of 2.0.0 that most users can
> > move to easily.
> >
> > Thanks,
> > Kevin
> >
> > On Jan 11, 2023 at 12:32:46, Joe Witt <jo...@gmail.com> wrote:
> >
> > > Kevin,
> > >
> > > Yeah we can do whatever we want as far as 'releases' of 2.0 that are
> > prior
> > > to us officially considering it 2.0/stable.
> > >
> > > That said - the migration tooling will be key to provide as we need to
> > make
> > > the bridge as solid and stable as possible to help someone move from
> 1.x
> > to
> > > 2.x. I dont know how related these two concepts (milestone releases
> and
> > > 1.x to 2.x ease really are).
> > >
> > > Thanks
> > >
> > > On Wed, Jan 11, 2023 at 10:27 AM Kevin Doran <kd...@apache.org>
> wrote:
> > >
> > > [hit the wrong keyboard shortcut, here is the rest of my thoughts]
> > >
> > >
> > > On this point from David:
> > >
> > >
> > > We may need to have a longer release candidate period, or more
> > incremental
> > >
> > > > fix releases
> > >
> > > > for the initial 2.0.0 release train, but I do not expect delaying a
> > 2.0.0
> > >
> > > > release for new features, as that is not part of the release goals.
> > >
> > > >
> > >
> > >
> > > Would the community benefit from one or more milestone releases of 2.0,
> > to
> > >
> > > allow for a wider group to run / live on the proposed 2.0 prior to
> > >
> > > releasing it as "stable"? I know we've never done a milestone release
> in
> > >
> > > the past, and I'm not sure what ASF guidance is on the topic, but if it
> > >
> > > could be beneficial we could look into that.
> > >
> > >
> > > Cheers,
> > >
> > > Kevin
> > >
> > >
> > > On Jan 11, 2023 at 12:22:43, Kevin Doran <kd...@apache.org> wrote:
> > >
> > >
> > > > I think this is a good, practical discussion.
> > >
> > > >
> > >
> > > > On the one hand, we can't put off 2.x any longer as we really need to
> > >
> > > > updated the minimum required Java to 11. Updating main development to
> > >
> > > > target 2.x feels like a good way drive progress on that along with
> the
> > >
> > > > other 2.0 goals.
> > >
> > > >
> > >
> > > > On the other hand, the concerns are valid: moving all development to
> > >
> > > > target 2.x puts the project at risk if we cannot release 2.0.0 on a
> > >
> > > > reasonable timeline. The restricted scope of 2.0 helps, but this
> stated
> > >
> > > > release goal feels risky to me:
> > >
> > > >
> > >
> > > > Implement Migration Tools for Upgrading Flows
> > >
> > > >
> > >
> > > >
> > >
> > > > - Implement automated migration where possible to remap properties
> > and
> > >
> > > > features
> > >
> > > > - Implement migration tools for manual conversion of XML
> > Templates
> > >
> > > > to JSON Flow Definitions
> > >
> > > > - Create documentation for manual steps necessary where
> > >
> > > > programmatic migration cannot be implemented
> > >
> > > > - NiFi 2.0 should be capable of starting with ghosted
> components
> > >
> > > > for removed Processors or Controller Services
> > >
> > > >
> > >
> > > >
> > >
> > > > Removing deprecated components should be fairly straightforward and
> > >
> > > quick,
> > >
> > > > but automating and documenting migration is a large effort.
> > >
> > > >
> > >
> > > > On this po
> > >
> > > >
> > >
> > > >
> > >
> > > > On Jan 10, 2023 at 09:32:31, Bryan Bende <bb...@gmail.com> wrote:
> > >
> > > >
> > >
> > > >> The plan as I understand it is not to diverge and create separate
> > >
> > > feature
> > >
> > > >> development on the 1.x line, so I would expect all PRs to continue
> to
> > be
> > >
> > > >> submitted only to main. We would release 1.x as needed with major
> bug
> > >
> > > >> fixes
> > >
> > > >> or critical security updates, and these would be cherry-picked
> and/or
> > >
> > > >> backported as necessary, mostly without the need for PRs, the same
> as
> > we
> > >
> > > >> would do if we were bringing fixes from main (1.20.0-SNAPSHOT) back
> > to a
> > >
> > > >> maintenance line like (1.19.x). For precedent, we followed this same
> > >
> > > >> approach going from the 0.x line to 1.0.0 and there wasn't any major
> > >
> > > >> issue.
> > >
> > > >>
> > >
> > > >> On Tue, Jan 10, 2023 at 7:07 AM Otto Fowler <
> ottobackwards@gmail.com>
> > >
> > > >> wrote:
> > >
> > > >>
> > >
> > > >> It was also mentioned in another thread that we need to have
> > agreement
> > >
> > > on
> > >
> > > >>
> > >
> > > >> our explicit strategy and support for 1.x going forward, did that
> > >
> > > happen?
> > >
> > > >>
> > >
> > > >>
> > >
> > > >> From: Otto Fowler <ot...@gmail.com> <
> ottobackwards@gmail.com>
> > >
> > > >>
> > >
> > > >> Reply: Otto Fowler <ot...@gmail.com> <
> ottobackwards@gmail.com
> > >
> > >
> > > >>
> > >
> > > >> Date: January 10, 2023 at 07:02:34
> > >
> > > >>
> > >
> > > >> To: dev@nifi.apache.org <de...@nifi.apache.org> <de...@nifi.apache.org>
> > >
> > > >>
> > >
> > > >> Subject: Re: [discuss] NiFi 1.20 and NiFi 2.0
> > >
> > > >>
> > >
> > > >>
> > >
> > > >> There needs to be an update to the contributing guide as to how to
> > >
> > > submit
> > >
> > > >>
> > >
> > > >> PRs to 1.x or 2.x etc.
> > >
> > > >>
> > >
> > > >>
> > >
> > > >> From: Joe Witt <jo...@apache.org> <jo...@apache.org>
> > >
> > > >>
> > >
> > > >> Reply: dev@nifi.apache.org <de...@nifi.apache.org> <
> dev@nifi.apache.org
> > >
> > >
> > > >>
> > >
> > > >> Date: January 9, 2023 at 15:53:16
> > >
> > > >>
> > >
> > > >> To: dev@nifi.apache.org <de...@nifi.apache.org> <de...@nifi.apache.org>
> > >
> > > >>
> > >
> > > >> Subject: [discuss] NiFi 1.20 and NiFi 2.0
> > >
> > > >>
> > >
> > > >>
> > >
> > > >> Team,
> > >
> > > >>
> > >
> > > >>
> > >
> > > >> As David mentioned in [1] following a successful NiFi 2.0 release
> goal
> > >
> > > >>
> > >
> > > >> planning - we are now going to start moving the 'main' line to be
> the
> > >
> > > NiFi
> > >
> > > >>
> > >
> > > >> 2.0 line which will allow for the key work to take place. We will
> also
> > >
> > > >>
> > >
> > > >> move niFi 1.x to its appropriate support line.
> > >
> > > >>
> > >
> > > >>
> > >
> > > >> It is also the case that we have nearly 100 JIRAs on NiFi 1.20 and
> we
> > >
> > > have
> > >
> > > >>
> > >
> > > >> work in there including security items so it is time to make a
> > release.
> > >
> > > >>
> > >
> > > >> The intent then is to initiate 1.20 and immediate after that change
> > >
> > > 'main'
> > >
> > > >>
> > >
> > > >> to 2.0.
> > >
> > > >>
> > >
> > > >>
> > >
> > > >> Going forward then all work on the 1.x line should be focused on
> > >
> > > >>
> > >
> > > >> maintaining existing features, dependencies, and helping 1.x users
> > >
> > > migrate
> > >
> > > >>
> > >
> > > >> to the 2.x line. Otherwise, new feature work will happen on 'main'
> as
> > it
> > >
> > > >>
> > >
> > > >> normally does and will come out in the NiFi 2.x release line.
> > >
> > > >>
> > >
> > > >>
> > >
> > > >> Please flag key outstanding items as we narrow down the release
> > >
> > > candidate
> > >
> > > >>
> > >
> > > >> for NiFi 1.20.
> > >
> > > >>
> > >
> > > >>
> > >
> > > >> Thanks
> > >
> > > >>
> > >
> > > >> Joe
> > >
> > > >>
> > >
> > > >>
> > >
> > > >> [1]
> https://lists.apache.org/thread/qo4vvdw46235y7vy2crcd6l4m11wl7jz
> > >
> > > >>
> > >
> > > >>
> > >
> > > >>
> > >
> > >
> > >
> >
>
Re: [discuss] NiFi 1.20 and NiFi 2.0
Posted by Joe Witt <jo...@gmail.com>.
Hello
We dont have a release schedule and never really have. What we do have is
7+ years of track record which shows who and what we do. We have done a
major release change before. We release every couple of months or so.
This is very manageable and the stated goals are scoped as such.
We have declared what our goals are for 2.0 and now it is time to simply
make it happen. We will keep 1.x moving along in the meantime but one of
these is going to get the ‘most’ energy as that is ultimately how all this
works.
Whatever is our ‘main’ line is where all PRs go by default and where people
work by default. The burden of maintaining these lines is very real and
will provide plenty of motivation to move 2.0 along rapidly. The true
notion of business as usual is where PRs land and who does the work to
review and merge them.
The commentary around production usage worries suggests ideas or views that
differ from the discussed, documented and now voted upon release goals.
Someone starting on 2.0 should enjoy stability similar to a NiFi 1.20
release. Existing users will be provided tooling/automation/docs to move
from 1.x to 2.x.
Milestones may serve as a valuable tool
for us. We can use them as we vet out migration tooling we put on the 1.x
line.
Do we need anything else for 1.20?
Thanks
On Wed, Jan 11, 2023 at 3:42 PM Adam Taft <ad...@adamtaft.com> wrote:
> This is really insightful and spot on ...
>
> Kevin wrote:
> > Good migration tooling will take a while to develop and test, and the
> core
> > contributors to that effort may not have sufficient variety of flows to
> > evaluate when the migration tools are "done" for the majority of the
> > community to have success upgrading to 2.x. A milestone release would
> allow
> > us get more feedback on migration over a longer period than the vote
> window
> > of an RC candidate.
>
> It's exactly this case, that an early 2.0 release might not have had time
> to fully work its way through existing production deployments, that's
> concerning. The pace and voting of an "RC" is much too short to get any
> quality feedback from users in the field.
>
> I think it's really smart to consider the "Milestone" release approach
> here. We release 2.0.0-M1, 2.0.0-M2, ... waiting an adequate amount of time
> for feedback. We can put these milestones on a calendar, as needed, so that
> feedback is required some 'x' number of weeks/months after each milestone.
>
> And to this end, I'd personally rather see us keep the 'main' branch
> current with the 1.x line _until_ we're ready and are satisfied with the
> end goals of the 2.0 release objectives. When the milestone releases have
> been completed and there's a comfort level with the 2.x line, it's at the
> point we'd isolate the 1.x line into its own branch and switch main over to
> the 2.x line.
>
> This is an attractive way of:
> a) continuing business-as-usual with the 1.x line
> b) making headway on the 2.x release milestones
> c) giving adequate time for feedback against the 2.0 milestones coming from
> the field
>
> I don't mind the known-unknowns. But it's really the unknown-unknowns that
> are going to drive a delay in the 2.0 release. I think it's smart to be
> able to get some of the unknowns ironed out before we finalize the 2.0
> release ceremony. The milestone approach really helps with that.
>
> /Adam
>
> On Wed, Jan 11, 2023 at 11:11 AM Kevin Doran <kd...@apache.org> wrote:
>
> > Sorry, Joe, I was not clear, and to be honest the two thoughts are
> somewhat
> > unrelated in my mind too :)
> >
> > I agree that good migration tooling is key. Otherwise, we risk users
> > staying on 1.x or creating a schism of 1.x and 2.x users.
> >
> > Good migration tooling will take a while to develop and test, and the
> core
> > contributors to that effort may not have sufficient variety of flows to
> > evaluate when the migration tools are "done" for the majority of the
> > community to have success upgrading to 2.x. A milestone release would
> allow
> > us get more feedback on migration over a longer period than the vote
> window
> > of an RC candidate.
> >
> > Perhaps we could continue to release from the 1.x line (including minor
> > releases with some features) until we are ready to drop the "milestone"
> > qualifier from 2.0.0, and only then put 1.x into maintenance-only status.
> > It would be the same proposal to move main to target 2.0.0-M1, but relax
> > restrictions for what can land on the 1.x branch and be open to a 1.21,
> > 1.22, etc. if 2.0.0 work takes longer than anticipated. For example,
> maybe
> > we would be open to landing new/backported processors on the 1.x branch,
> > but not core framework features or API changes.
> >
> > This might not be necessary, but I think it is fair that saying "no new
> > features on 1.x" and also "no new features in 2.0.0" puts the project in
> a
> > rough place if 2.0.0 takes longer than a few months, so if we go that
> > route, we need to commit to a quick release of 2.0.0 that most users can
> > move to easily.
> >
> > Thanks,
> > Kevin
> >
> > On Jan 11, 2023 at 12:32:46, Joe Witt <jo...@gmail.com> wrote:
> >
> > > Kevin,
> > >
> > > Yeah we can do whatever we want as far as 'releases' of 2.0 that are
> > prior
> > > to us officially considering it 2.0/stable.
> > >
> > > That said - the migration tooling will be key to provide as we need to
> > make
> > > the bridge as solid and stable as possible to help someone move from
> 1.x
> > to
> > > 2.x. I dont know how related these two concepts (milestone releases
> and
> > > 1.x to 2.x ease really are).
> > >
> > > Thanks
> > >
> > > On Wed, Jan 11, 2023 at 10:27 AM Kevin Doran <kd...@apache.org>
> wrote:
> > >
> > > [hit the wrong keyboard shortcut, here is the rest of my thoughts]
> > >
> > >
> > > On this point from David:
> > >
> > >
> > > We may need to have a longer release candidate period, or more
> > incremental
> > >
> > > > fix releases
> > >
> > > > for the initial 2.0.0 release train, but I do not expect delaying a
> > 2.0.0
> > >
> > > > release for new features, as that is not part of the release goals.
> > >
> > > >
> > >
> > >
> > > Would the community benefit from one or more milestone releases of 2.0,
> > to
> > >
> > > allow for a wider group to run / live on the proposed 2.0 prior to
> > >
> > > releasing it as "stable"? I know we've never done a milestone release
> in
> > >
> > > the past, and I'm not sure what ASF guidance is on the topic, but if it
> > >
> > > could be beneficial we could look into that.
> > >
> > >
> > > Cheers,
> > >
> > > Kevin
> > >
> > >
> > > On Jan 11, 2023 at 12:22:43, Kevin Doran <kd...@apache.org> wrote:
> > >
> > >
> > > > I think this is a good, practical discussion.
> > >
> > > >
> > >
> > > > On the one hand, we can't put off 2.x any longer as we really need to
> > >
> > > > updated the minimum required Java to 11. Updating main development to
> > >
> > > > target 2.x feels like a good way drive progress on that along with
> the
> > >
> > > > other 2.0 goals.
> > >
> > > >
> > >
> > > > On the other hand, the concerns are valid: moving all development to
> > >
> > > > target 2.x puts the project at risk if we cannot release 2.0.0 on a
> > >
> > > > reasonable timeline. The restricted scope of 2.0 helps, but this
> stated
> > >
> > > > release goal feels risky to me:
> > >
> > > >
> > >
> > > > Implement Migration Tools for Upgrading Flows
> > >
> > > >
> > >
> > > >
> > >
> > > > - Implement automated migration where possible to remap properties
> > and
> > >
> > > > features
> > >
> > > > - Implement migration tools for manual conversion of XML
> > Templates
> > >
> > > > to JSON Flow Definitions
> > >
> > > > - Create documentation for manual steps necessary where
> > >
> > > > programmatic migration cannot be implemented
> > >
> > > > - NiFi 2.0 should be capable of starting with ghosted
> components
> > >
> > > > for removed Processors or Controller Services
> > >
> > > >
> > >
> > > >
> > >
> > > > Removing deprecated components should be fairly straightforward and
> > >
> > > quick,
> > >
> > > > but automating and documenting migration is a large effort.
> > >
> > > >
> > >
> > > > On this po
> > >
> > > >
> > >
> > > >
> > >
> > > > On Jan 10, 2023 at 09:32:31, Bryan Bende <bb...@gmail.com> wrote:
> > >
> > > >
> > >
> > > >> The plan as I understand it is not to diverge and create separate
> > >
> > > feature
> > >
> > > >> development on the 1.x line, so I would expect all PRs to continue
> to
> > be
> > >
> > > >> submitted only to main. We would release 1.x as needed with major
> bug
> > >
> > > >> fixes
> > >
> > > >> or critical security updates, and these would be cherry-picked
> and/or
> > >
> > > >> backported as necessary, mostly without the need for PRs, the same
> as
> > we
> > >
> > > >> would do if we were bringing fixes from main (1.20.0-SNAPSHOT) back
> > to a
> > >
> > > >> maintenance line like (1.19.x). For precedent, we followed this same
> > >
> > > >> approach going from the 0.x line to 1.0.0 and there wasn't any major
> > >
> > > >> issue.
> > >
> > > >>
> > >
> > > >> On Tue, Jan 10, 2023 at 7:07 AM Otto Fowler <
> ottobackwards@gmail.com>
> > >
> > > >> wrote:
> > >
> > > >>
> > >
> > > >> It was also mentioned in another thread that we need to have
> > agreement
> > >
> > > on
> > >
> > > >>
> > >
> > > >> our explicit strategy and support for 1.x going forward, did that
> > >
> > > happen?
> > >
> > > >>
> > >
> > > >>
> > >
> > > >> From: Otto Fowler <ot...@gmail.com> <
> ottobackwards@gmail.com>
> > >
> > > >>
> > >
> > > >> Reply: Otto Fowler <ot...@gmail.com> <
> ottobackwards@gmail.com
> > >
> > >
> > > >>
> > >
> > > >> Date: January 10, 2023 at 07:02:34
> > >
> > > >>
> > >
> > > >> To: dev@nifi.apache.org <de...@nifi.apache.org> <de...@nifi.apache.org>
> > >
> > > >>
> > >
> > > >> Subject: Re: [discuss] NiFi 1.20 and NiFi 2.0
> > >
> > > >>
> > >
> > > >>
> > >
> > > >> There needs to be an update to the contributing guide as to how to
> > >
> > > submit
> > >
> > > >>
> > >
> > > >> PRs to 1.x or 2.x etc.
> > >
> > > >>
> > >
> > > >>
> > >
> > > >> From: Joe Witt <jo...@apache.org> <jo...@apache.org>
> > >
> > > >>
> > >
> > > >> Reply: dev@nifi.apache.org <de...@nifi.apache.org> <
> dev@nifi.apache.org
> > >
> > >
> > > >>
> > >
> > > >> Date: January 9, 2023 at 15:53:16
> > >
> > > >>
> > >
> > > >> To: dev@nifi.apache.org <de...@nifi.apache.org> <de...@nifi.apache.org>
> > >
> > > >>
> > >
> > > >> Subject: [discuss] NiFi 1.20 and NiFi 2.0
> > >
> > > >>
> > >
> > > >>
> > >
> > > >> Team,
> > >
> > > >>
> > >
> > > >>
> > >
> > > >> As David mentioned in [1] following a successful NiFi 2.0 release
> goal
> > >
> > > >>
> > >
> > > >> planning - we are now going to start moving the 'main' line to be
> the
> > >
> > > NiFi
> > >
> > > >>
> > >
> > > >> 2.0 line which will allow for the key work to take place. We will
> also
> > >
> > > >>
> > >
> > > >> move niFi 1.x to its appropriate support line.
> > >
> > > >>
> > >
> > > >>
> > >
> > > >> It is also the case that we have nearly 100 JIRAs on NiFi 1.20 and
> we
> > >
> > > have
> > >
> > > >>
> > >
> > > >> work in there including security items so it is time to make a
> > release.
> > >
> > > >>
> > >
> > > >> The intent then is to initiate 1.20 and immediate after that change
> > >
> > > 'main'
> > >
> > > >>
> > >
> > > >> to 2.0.
> > >
> > > >>
> > >
> > > >>
> > >
> > > >> Going forward then all work on the 1.x line should be focused on
> > >
> > > >>
> > >
> > > >> maintaining existing features, dependencies, and helping 1.x users
> > >
> > > migrate
> > >
> > > >>
> > >
> > > >> to the 2.x line. Otherwise, new feature work will happen on 'main'
> as
> > it
> > >
> > > >>
> > >
> > > >> normally does and will come out in the NiFi 2.x release line.
> > >
> > > >>
> > >
> > > >>
> > >
> > > >> Please flag key outstanding items as we narrow down the release
> > >
> > > candidate
> > >
> > > >>
> > >
> > > >> for NiFi 1.20.
> > >
> > > >>
> > >
> > > >>
> > >
> > > >> Thanks
> > >
> > > >>
> > >
> > > >> Joe
> > >
> > > >>
> > >
> > > >>
> > >
> > > >> [1]
> https://lists.apache.org/thread/qo4vvdw46235y7vy2crcd6l4m11wl7jz
> > >
> > > >>
> > >
> > > >>
> > >
> > > >>
> > >
> > >
> > >
> >
>
Re: [discuss] NiFi 1.20 and NiFi 2.0
Posted by Kevin Doran <kd...@apache.org>.
To add to what Mark said, please make sure when merging and cherry-picking
bug fixes, to resole the Jira ticket include both the 1.x and 2.x versions
in the “fix versions” field (or just the correct one if the bug fix is only
applied to one of the branches).
Thanks,
kevin
On Feb 13, 2023 at 18:06:56, Mark Payne <ma...@hotmail.com> wrote:
> Otto,
>
> Good points. Generally, I think we don’t need two PRs. The committer
> should just merge to main and then cherry-pick to the support/nifi-1.x
> branch.
>
> In the event that there are conflicts, my suggestion would be:
>
> The committer should make a reasonable effort to resolve the issue. I.e.,
> if it doesn’t apply cleanly but it’s a 3-line change that can just be
> copied from one branch and pasted to the other, then go ahead and make that
> change.
> If the PR is substantial enough that it cannot be easily merged, the
> committer should notify the contributor that it cannot be merged to the 1.x
> branch. In that event, it would be up to the committer (or someone else) to
> submit a new PR for the 1.x baseline. If that doesn’t happen, the fix would
> be merged only into the 2.x baseline.
>
> I agree it’s probably a good idea, if opening a PR targeted explicitly to
> the 1.x baseline to note that in the title. Perhaps denoting "[1.x only]”
> or something along those lines. But GitHub should denote which upstream
> branch it’s intended to be merged to. So it will be up to the dilligence of
> committers to ensure that we’re merging to the right branch.
>
> Thanks
> -Mark
>
>
> On Feb 11, 2023, at 11:16 AM, Otto Fowler <ot...@gmail.com> wrote:
>
> Thanks Mark,
> Can you give an example of best practices cherry picking? This cherry
> picked commit ( and any other re-work required to make it work ) will be a
> new PR then? Or is that only if it doesn’t go in clean?
>
> Contributor:
> - Pull Request
> Committer:
> - Merge
> - Cherry Pick to new PR branch off of support/nifi-1.x
> - possible fixups
> - push a new PR
>
> Should the support/nifi-1.x PRs have a standard name format WRT the
> original? Should the name be the same as the subject of the other, or
> should it be the PR#?
>
>
>
> From: Mark Payne <ma...@hotmail.com>> <
> markap14@hotmail.com<ma...@hotmail.com>>
> Reply: dev@nifi.apache.org<ma...@nifi.apache.org> <
> dev@nifi.apache.org<ma...@nifi.apache.org>> <dev@nifi.apache.org
> <ma...@nifi.apache.org>>
> Date: February 10, 2023 at 11:51:28
> To: dev@nifi.apache.org<ma...@nifi.apache.org> <dev@nifi.apache.org
> <ma...@nifi.apache.org>> <dev@nifi.apache.org<mailto:
> dev@nifi.apache.org>>
> Subject: Re: [discuss] NiFi 1.20 and NiFi 2.0
>
> Thanks Joe!
>
> So at this point, we now have the 1.x and the 2.x line diverging. As a
> result, we will want to ensure that we keep the 1.x line up-to-date with
> any bug fixes.
> Trying to cherry-pick over just the necessary bug fixes for a release down
> the road has the potential to be a nightmare, since some PRs may not apply
> cleanly.
>
> In order to avoid this: committers please ensure that for any bug fixes, if
> you merge to the “main” branch that you also cherry-pick the commit to the
> “support/nifi-1.x” branch.
> If you’ve not dealt with support branches in the past, there’s one small
> gotcha, as the branch name has a slash (/) in it. So to checkout the
> support branch, you’ll use:
>
> git checkout origin/support/nifi-1.x
>
> And not just “git checkout support/nifi-1.x”
>
> Then, you can create a local branch:
>
> git checkout -b support/nifi-1.x
>
> And after cherry-picking the necessary commits, you can push:
>
> git push -u origin support/nifi-1.x
>
> This will help to ensure that we keep a clean support branch so that we’re
> able to make a release any time if the need arises, which will include all
> of the necessary bug fixes.
>
> Thanks
> -Mark
>
>
>
> On Feb 9, 2023, at 5:43 PM, Joe Witt <jo...@gmail.com> wrote:
> br/>> Team, <
> br/>> Alrighty Apache NiFFi 1.20.0 is official!
> br/>> As a result our Apache NiFFi git branch 'main' is now officially
> set up
> as our go forward line and is 2.0.0-SNAPSHOT as of now.
> br/>> To continue releases as needed for the 1.x linee such as
> 1.21.0-SNAPSHOT as would come next we now have 'support/nifi-1.x'. It
> is already setup for 1.21.0-SNAPSHOT versioning. We will need at
> least one such release on that line for migration tooling to help
> users move from that 1.x line to the 2.x line but depending on how
> long 2.0 takes to settle we may have more. Whatever it takes.
> br/>> Exciting times!!
> br/>> Thanks <
> Joe
> br/>> On Tue, FFeb 7, 2023 at 8:40 AM Joe Witt <joe.witt@gmail.com<mailto:
> joe.witt@gmail.com>>
> wrote:
> br/>>> Team <
> br/>>> As you commit to the main line now pleease start using whatever
> the latest relevant feature release is on the 2.x line which right now
> would be 2.0.0. If we find we need to do a 1.21 and so on we'll pull those
> commits down as we would for any other mx release in the past but the
> 'main' line becomes 2.0.0 now.
> br/>>> Thanks <
> Joe
> br/>>> On Mon, FFeb 6, 2023 at 10:44 AM Joe Witt <joe.witt@gmail.com
> <ma...@gmail.com>>
> wrote:
> br/>>>> Team, <
> br/>>>> I think we are there!! Going to kick out RC1 now.
> br/>>>> Thanks <
> br/>>>> On Mon, Jan 30, 2023 at 11:29 AM Joe Witt <joe.witt@gmail.com
> <ma...@gmail.com>>
> wrote:
> br/>>>>> Team, <
> br/>>>>> As you can see I've noot kicked off the RC yet. Many bug
> fixes/dependency updates are happening. Ideally we'll wait until nar Maven
> plugin goes and we're trying to fix some nifi registry/nifi interaction
> issues as well. Still will get the RC out as soon as we can.
> br/>>>>> Thanks <
> br/>>>>> On Mon, Jan 23, 2023 aat 11:12 AM Joe Witt <joe.witt@gmail.com
> <ma...@gmail.com>>
> wrote:
> br/>>>>>> Hello <
> br/>>>>>> Here is a goodd picture of what the 1.20 RC looks like.
> I've tagged several JIRAs today to ensure we get them in. A theme is really
> around stabilizing nifi/nifi-registry integration as we're seeing
> substantial uptick in usage and thus various community reported findings.
> We'll get that quite smooth with these included.
> br/>>>>>> https://issues<https://issues/>..
> apache.org/jira/projects/NIFI/versions/12352581<
> http://apache.org/jira/projects/NIFI/versions/12352581>
> br/>>>>>> Thanks <
> br/>>>>>> On Mon, Jan 233, 2023 at 8:50 AM Joe Witt <
> joe.witt@gmail.com<ma...@gmail.com>> wrote:
> br/>>>>>>> Team, <
> br/>>>>>>> I'm goiing through the outstanding JIRAs/PRs and flagging
> which look like they should be 'must have' for 1.20 and then will work the
> RC as soon as those land.
> br/>>>>>>> Hopefullly have the RC up within a day or two but we'll
> see how these land as some have review comments pending action.
> br/>>>>>>> Thanks
> br/>>>>>>> On Thu,, Jan 12, 2023 at 2:53 AM Isha Lamboo <
> isha.lamboo@virtualsciences.nl<ma...@virtualsciences.nl>>
> wrote:
> br/>>>>>>>>; Hi all,
> br/>>>>>>>>; I would like to contribute to the migration tooling
> (mostly testing I suppose) when that comes up.
> br/>>>>>>>>; My team's largest client has a completely
> template-based pipeline with external scripts replacing variable values
> before deploying to target clusters, so we've already started looking at
> this when the goals for 2.0 were discussed and approved. The migration to
> flowdefinitions and parameters is quite complex and we've hit several
> blockers when we tried to implement a direct translation.
> br/>>>>>>>>; I expect that any time I spend helping to improve the
> tooling will pay off handsomely for our clients.
> br/>>>>>>>>; Regards,
> br/>>>>>>>>; Isha
> br/>>>>>>>>; -----Oorspronkelijk bericht-----
> Van: Adam Taft <ad...@adamtaft.com>>
> Verzonden: woensdag 11 januari 2023 23:42
> Aan: dev@nifi.apache.org<ma...@nifi.apache.org>
> Onderwerp: Re: [discuss] NiFi 1.20 and NiFi 2.0
> br/>>>>>>>>; This is really insightful and spot on ...
> br/>>>>>>>>; Kevin wrote:
> Good migration tooling will take a while to develop and test, and
> the
> core contributors to that effort may not have sufficient variety
> of
> flows to evaluate when the migration tools are "done" for the
> majority
> of the community to have success upgrading to 2.x. A milestone
> release
> would
> allow
> us get more feedback on migration over a longer period than the
> vote
> window
> of an RC candidate.
> br/>>>>>>>>; It's exactly this case, that an early 2.0 release
> might not have had time to fully work its way through existing production
> deployments, that's concerning. The pace and voting of an "RC" is much too
> short to get any quality feedback from users in the field.
> br/>>>>>>>>; I think it's really smart to consider the "Milestone"
> release approach here. We release 2.0.0-M1, 2.0.0-M2, ... waiting an
> adequate amount of time for feedback. We can put these milestones on a
> calendar, as needed, so that feedback is required some 'x' number of
> weeks/months after each milestone.
> br/>>>>>>>>; And to this end, I'd personally rather see us keep the
> 'main' branch current with the 1.x line _until_ we're ready and are
> satisfied with the end goals of the 2.0 release objectives. When the
> milestone releases have been completed and there's a comfort level with the
> 2.x line, it's at the point we'd isolate the 1.x line into its own branch
> and switch main over to the 2.x line.
> br/>>>>>>>>; This is an attractive way of:
> a) continuing business-as-usual with the 1.x line
> b) making headway on the 2.x release milestones
> c) giving adequate time for feedback against the 2.0 milestones
> coming from the field
> br/>>>>>>>>; I don't mind the known-unknowns. But it's really the
> unknown-unknowns that are going to drive a delay in the 2.0 release. I
> think it's smart to be able to get some of the unknowns ironed out before
> we finalize the 2.0 release ceremony. The milestone approach really helps
> with that.
> br/>>>>>>>>; /Adam
> br/>>>>>>>>; On Wed, Jan 11, 2023 at 11:11 AM Kevin Doran <
> kdoran@apache.org<ma...@apache.org>> wrote:
> br/>>>>>>>>;> Sorry, Joe, I was not clear, and to be honest the two
> thoughts are
> somewhat unrelated in my mind too :)
> br/>>>>>>>;>> I agree that good migration tooling is key.
> Otherwise, we risk users
> staying on 1.x or creating a schism of 1.x and 2.x users.
> br/>>>>>>>;>> Good migration tooling will take a while to develop
> and test, and the
> core contributors to that effort may not have sufficient variety
> of
> flows to evaluate when the migration tools are "done" for the
> majority
> of the community to have success upgrading to 2.x. A milestone
> release
> would allow us get more feedback on migration over a longer period
> than the vote window of an RC candidate.
> br/>>>>>>>;>> Perhaps we could continue to release from the 1.x
> line (including
> minor releases with some features) until we are ready to drop the
> "milestone"
> qualifier from 2.0.0, and only then put 1.x into maintenance-only
> status.
> It would be the same proposal to move main to target 2.0.0-M1, but
> relax restrictions for what can land on the 1.x branch and be open
> to
> a 1.21, 1.22, etc. if 2.0.0 work takes longer than anticipated.
> For
> example, maybe we would be open to landing new/backported
> processors
> on the 1.x branch, but not core framework features or API changes.
> br/>>>>>>>;>> This might not be necessary, but I think it is fair
> that saying "no
> new features on 1.x" and also "no new features in 2.0.0" puts the
> project in a rough place if 2.0.0 takes longer than a few months,
> so
> if we go that route, we need to commit to a quick release of 2.0.0
> that most users can move to easily.
> br/>>>>>>>;>> Thanks,
> Kevin
> br/>>>>>>>;>> On Jan 11, 2023 at 12:32:46, Joe Witt <
> joe.witt@gmail.com<ma...@gmail.com>> wrote:
> br/>>>>>>>;>>> Kevin,
> br/>>>>>>;>>>> Yeah we can do whatever we want as far as
> 'releases' of 2.0 that are
> prior
> to us officially considering it 2.0/stable.
> br/>>>>>>;>>>> That said - the migration tooling will be key to
> provide as we need
> to
> make
> the bridge as solid and stable as possible to help someone move
> from
> 1.x
> to
> 2.x. I dont know how related these two concepts (milestone
> releases
> and 1.x to 2.x ease really are).
> br/>>>>>>;>>>> Thanks
> br/>>>>>>;>>>> On Wed, Jan 11, 2023 at 10:27 AM Kevin Doran <
> kdoran@apache.org<ma...@apache.org>> wrote:
> br/>>>>>>;>>>> [hit the wrong keyboard shortcut, here is the rest
> of my thoughts]
> br/>>>>>>;>>>> br/>>>>>>>>>> On this pooint from David:
> br/>>>>>>;>>>> br/>>>>>>>>>> We may neeed to have a longer
> release candidate period, or more
> incremental
> br/>>>>>>;>>>>> fix releases
> br/>>>>>>;>>>>> for the initial 2.0.0 release train, but I do not
> expect delaying
> a
> 2.0.0
> br/>>>>>>;>>>>> release for new features, as that is not part of
> the release goals.
> br/>>>>>>;>>>>> br/>>>>>>>>>> br/>>>>>>>>>> br/>>>>>>>>>> Would
> the community benefit from one or more milestone releases of
> 2.0,
> to
> br/>>>>>>;>>>> allow for a wider group to run / live on the
> proposed 2.0 prior to
> br/>>>>>>;>>>> releasing it as "stable"? I know we've never done
> a milestone
> release in
> br/>>>>>>;>>>> the past, and I'm not sure what ASF guidance is on
> the topic, but if
> it
> br/>>>>>>;>>>> could be beneficial we could look into that.
> br/>>>>>>;>>>> br/>>>>>>>>>> Cheers, <
> br/>>>>>>t;>>>> Kevin
> br/>>>>>>;>>>> br/>>>>>>>>>> On Jan 11,, 2023 at 12:22:43, Kevin
> Doran <kd...@apache.org>> wrote:
> br/>>>>>>;>>>> br/>>>>>>>>>>> I thinnk this is a good, practical
> discussion.
> br/>>>>>>;>>>>> br/>>>>>>>>>> br/>>>>>>>>>>> On the one hand, we
> can''t put off 2.x any longer as we really need
> to
> br/>>>>>>;>>>>> updated the minimum required Java to 11. Updating
> main development
> to
> br/>>>>>>;>>>>> target 2.x feels like a good way drive progress
> on that along with
> the
> br/>>>>>>;>>>>> other 2.0 goals.
> br/>>>>>>;>>>>> br/>>>>>>>>>> br/>>>>>>>>>>> On the other hand,
> the cconcerns are valid: moving all development
> to
> br/>>>>>>;>>>>> target 2.x puts the project at risk if we cannot
> release 2.0.0 on
> a
> br/>>>>>>;>>>>> reasonable timeline. The restricted scope of 2.0
> helps, but this
> stated
> br/>>>>>>;>>>>> release goal feels risky to me:
> br/>>>>>>;>>>>> br/>>>>>>>>>> br/>>>>>>>>>>> Implement Migration
> Toolls for Upgrading Flows
> br/>>>>>>;>>>>> br/>>>>>>>>>> br/>>>>>>>>>>> br/>>>>>>>>>>
> br/>>>>>>>>>>> - Implement automated migration where possible to remap
> properties
> and
> br/>>>>>>;>>>>> features
> br/>>>>>>;>>>>> - Implement migration tools for manual conversion
> of XML
> Templates
> br/>>>>>>;>>>>> to JSON Flow Definitions
> br/>>>>>>;>>>>> - Create documentation for manual steps necessary
> where
> br/>>>>>>;>>>>> programmatic migration cannot be implemented
> br/>>>>>>;>>>>> - NiFi 2.0 should be capable of starting with
> ghosted
> components
> br/>>>>>>;>>>>> for removed Processors or Controller Services
> br/>>>>>>;>>>>> br/>>>>>>>>>> br/>>>>>>>>>>> br/>>>>>>>>>>
> br/>>>>>>>>>>> Removing deprecated compponents should be fairly
> straightforward
> and
> br/>>>>>>;>>>> quick,
> br/>>>>>>;>>>>> but automating and documenting migration is a
> large effort.
> br/>>>>>>;>>>>> br/>>>>>>>>>> br/>>>>>>>>>>> On this po <
> br/>>>>>>t;>>>>> br/>>>>>>>>>> br/>>>>>>>>>>> br/>>>>>>>>>>
> br/>>>>>>>>>>> On Jan 10, 2023 at 09:322:31, Bryan Bende <bbende@gmail.com
> <ma...@gmail.com>>
> wrote:
> br/>>>>>>;>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> The plan as I
> undersstand it is not to diverge and create separate
> br/>>>>>>;>>>> feature
> br/>>>>>>;>>>>>> development on the 1.x line, so I would expect
> all PRs to
> continue to
> be
> br/>>>>>>;>>>>>> submitted only to main. We would release 1.x as
> needed with major
> bug
> br/>>>>>>;>>>>>> fixes
> br/>>>>>>;>>>>>> or critical security updates, and these would be
> cherry-picked
> and/or
> br/>>>>>>;>>>>>> backported as necessary, mostly without the need
> for PRs, the
> same as
> we
> br/>>>>>>;>>>>>> would do if we were bringing fixes from main
> (1.20.0-SNAPSHOT)
> back
> to a
> br/>>>>>>;>>>>>> maintenance line like (1.19.x). For precedent,
> we followed this
> same
> br/>>>>>>;>>>>>> approach going from the 0.x line to 1.0.0 and
> there wasn't any
> major
> br/>>>>>>;>>>>>> issue.
> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> On Tue, Jan 10,
> 20233 at 7:07 AM Otto Fowler
> <ot...@gmail.com>>
> br/>>>>>>;>>>>>> wrote:
> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> It was also
> mentioneed in another thread that we need to have
> agreement
> br/>>>>>>;>>>> on
> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> our explicit
> strateggy and support for 1.x going forward, did that
> br/>>>>>>;>>>> happen?
> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> br/>>>>>>>>>>
> br/>>>>>>>>>>>> FFrom: Otto Fowler <ottobackwards@gmail.com<mailto:
> ottobackwards@gmail.com>>
> <ot...@gmail.com>>
> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> Reply: Otto
> FFowler <ot...@gmail.com>>
> <ot...@gmail.com>
> br/>>>>>>;>>>> br/>>>>>>>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>>
> Date: January 10, 20023 at 07:02:34
> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> To: dev@@
> nifi.apache.org<http://nifi.apache.org/> <dev@nifi.apache.org<mailto:
> dev@nifi.apache.org>>
> <de...@nifi.apache.org>>
> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> Subject: Re:
> [[discuss] NiFi 1.20 and NiFi 2.0
> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> br/>>>>>>>>>>
> br/>>>>>>>>>>>> There needs to be ann update to the contributing guide as
> to how
> to
> br/>>>>>>;>>>> submit
> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> PRs to 1.x or 2.x
> ettc.
> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> br/>>>>>>>>>>
> br/>>>>>>>>>>>> FFrom: Joe Witt <joewitt@apache.org<mailto:
> joewitt@apache.org>> <jo...@apache.org>>
> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> Reply: dev@@
> nifi.apache.org<http://nifi.apache.org/> <dev@nifi.apache.org<mailto:
> dev@nifi.apache.org>>
> <de...@nifi.apache.org>
> br/>>>>>>;>>>> br/>>>>>>>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>>
> Date: January 9, 20223 at 15:53:16
> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> To: dev@@
> nifi.apache.org<http://nifi.apache.org/> <dev@nifi.apache.org<mailto:
> dev@nifi.apache.org>>
> <de...@nifi.apache.org>>
> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> Subject:
> [[discuss] NiFi 1.20 and NiFi 2.0
> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> br/>>>>>>>>>>
> br/>>>>>>>>>>>> Team, <
> br/>>>>>>t;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> br/>>>>>>>>>>
> br/>>>>>>>>>>>> As David mentioned iin [1] following a successful NiFi 2.0
> release
> goal
> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> planning - we are
> noow going to start moving the 'main' line to be
> the
> br/>>>>>>;>>>> NiFi
> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> 2.0 line which
> will allow for the key work to take place. We will
> also
> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> move niFFi 1.x to
> its appropriate support line.
> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> br/>>>>>>>>>>
> br/>>>>>>>>>>>> It is also the case that we have nearly 100 JIRAs on NiFi
> 1.20
> and we
> br/>>>>>>;>>>> have
> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> work in there
> includding security items so it is time to make a
> release.
> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> The intent then is
> tto initiate 1.20 and immediate after that
> change
> br/>>>>>>;>>>> 'main'
> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> to 2.0. <
> br/>>>>>>t;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> br/>>>>>>>>>>
> br/>>>>>>>>>>>> Going forward then aall work on the 1.x line should be
> focused on
> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> maintaining
> existingg features, dependencies, and helping 1.x
> users
> br/>>>>>>;>>>> migrate
> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> to the 2.x line.
> Othherwise, new feature work will happen on
> 'main' as
> it
> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> normally does and
> wiill come out in the NiFi 2.x release line.
> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> br/>>>>>>>>>>
> br/>>>>>>>>>>>> Please flag key outsstanding items as we narrow down the
> release
> br/>>>>>>;>>>> candidate
> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> for NiFFi 1.20.
> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> br/>>>>>>>>>>
> br/>>>>>>>>>>>> Thanks <
> br/>>>>>>t;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> Joe <
> br/>>>>>>t;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> br/>>>>>>>>>>
> br/>>>>>>>>>>>> [[1]
>
> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2
> Flists.apache.org%2Fthread%2Fqo4vvdw46235y7vy2crcd6l4m11wl7jz&dat
>
> a=05%7C01%7Cisha.lamboo%40virtualsciences.nl%7Ccbea974a2c1f479d48
>
>
> 9d08daf42521f1%7C21429da9e4ad45f99a6fcd126a64274b%7C0%7C0%7C63809
>
> 0737572228694%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjo
>
>
> iV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=zSqOK9zZPqXxLuxwo0QcqKGEAc7aXjfnnm4i%2BQt2B98%3D&reserved=0
>
> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> br/>>>>>>>>>>
> br/>>>>>>>>>>>> br/>>>>>>>>>> br/>>>>>>>>>> br/>>>>>>>>>> br/>>>>>>>>> br/>
> <
>
>
Re: [discuss] NiFi 1.20 and NiFi 2.0
Posted by Mark Payne <ma...@hotmail.com>.
Otto,
Good points. Generally, I think we don’t need two PRs. The committer should just merge to main and then cherry-pick to the support/nifi-1.x branch.
In the event that there are conflicts, my suggestion would be:
The committer should make a reasonable effort to resolve the issue. I.e., if it doesn’t apply cleanly but it’s a 3-line change that can just be copied from one branch and pasted to the other, then go ahead and make that change.
If the PR is substantial enough that it cannot be easily merged, the committer should notify the contributor that it cannot be merged to the 1.x branch. In that event, it would be up to the committer (or someone else) to submit a new PR for the 1.x baseline. If that doesn’t happen, the fix would be merged only into the 2.x baseline.
I agree it’s probably a good idea, if opening a PR targeted explicitly to the 1.x baseline to note that in the title. Perhaps denoting "[1.x only]” or something along those lines. But GitHub should denote which upstream branch it’s intended to be merged to. So it will be up to the dilligence of committers to ensure that we’re merging to the right branch.
Thanks
-Mark
On Feb 11, 2023, at 11:16 AM, Otto Fowler <ot...@gmail.com> wrote:
Thanks Mark,
Can you give an example of best practices cherry picking? This cherry
picked commit ( and any other re-work required to make it work ) will be a
new PR then? Or is that only if it doesn’t go in clean?
Contributor:
- Pull Request
Committer:
- Merge
- Cherry Pick to new PR branch off of support/nifi-1.x
- possible fixups
- push a new PR
Should the support/nifi-1.x PRs have a standard name format WRT the
original? Should the name be the same as the subject of the other, or
should it be the PR#?
From: Mark Payne <ma...@hotmail.com>> <ma...@hotmail.com>>
Reply: dev@nifi.apache.org<ma...@nifi.apache.org> <de...@nifi.apache.org>> <de...@nifi.apache.org>>
Date: February 10, 2023 at 11:51:28
To: dev@nifi.apache.org<ma...@nifi.apache.org> <de...@nifi.apache.org>> <de...@nifi.apache.org>>
Subject: Re: [discuss] NiFi 1.20 and NiFi 2.0
Thanks Joe!
So at this point, we now have the 1.x and the 2.x line diverging. As a
result, we will want to ensure that we keep the 1.x line up-to-date with
any bug fixes.
Trying to cherry-pick over just the necessary bug fixes for a release down
the road has the potential to be a nightmare, since some PRs may not apply
cleanly.
In order to avoid this: committers please ensure that for any bug fixes, if
you merge to the “main” branch that you also cherry-pick the commit to the
“support/nifi-1.x” branch.
If you’ve not dealt with support branches in the past, there’s one small
gotcha, as the branch name has a slash (/) in it. So to checkout the
support branch, you’ll use:
git checkout origin/support/nifi-1.x
And not just “git checkout support/nifi-1.x”
Then, you can create a local branch:
git checkout -b support/nifi-1.x
And after cherry-picking the necessary commits, you can push:
git push -u origin support/nifi-1.x
This will help to ensure that we keep a clean support branch so that we’re
able to make a release any time if the need arises, which will include all
of the necessary bug fixes.
Thanks
-Mark
On Feb 9, 2023, at 5:43 PM, Joe Witt <jo...@gmail.com> wrote:
br/>> Team, <
br/>> Alrighty Apache NiFFi 1.20.0 is official!
br/>> As a result our Apache NiFFi git branch 'main' is now officially
set up
as our go forward line and is 2.0.0-SNAPSHOT as of now.
br/>> To continue releases as needed for the 1.x linee such as
1.21.0-SNAPSHOT as would come next we now have 'support/nifi-1.x'. It
is already setup for 1.21.0-SNAPSHOT versioning. We will need at
least one such release on that line for migration tooling to help
users move from that 1.x line to the 2.x line but depending on how
long 2.0 takes to settle we may have more. Whatever it takes.
br/>> Exciting times!!
br/>> Thanks <
Joe
br/>> On Tue, FFeb 7, 2023 at 8:40 AM Joe Witt <jo...@gmail.com>>
wrote:
br/>>> Team <
br/>>> As you commit to the main line now pleease start using whatever
the latest relevant feature release is on the 2.x line which right now
would be 2.0.0. If we find we need to do a 1.21 and so on we'll pull those
commits down as we would for any other mx release in the past but the
'main' line becomes 2.0.0 now.
br/>>> Thanks <
Joe
br/>>> On Mon, FFeb 6, 2023 at 10:44 AM Joe Witt <jo...@gmail.com>>
wrote:
br/>>>> Team, <
br/>>>> I think we are there!! Going to kick out RC1 now.
br/>>>> Thanks <
br/>>>> On Mon, Jan 30, 2023 at 11:29 AM Joe Witt <jo...@gmail.com>>
wrote:
br/>>>>> Team, <
br/>>>>> As you can see I've noot kicked off the RC yet. Many bug
fixes/dependency updates are happening. Ideally we'll wait until nar Maven
plugin goes and we're trying to fix some nifi registry/nifi interaction
issues as well. Still will get the RC out as soon as we can.
br/>>>>> Thanks <
br/>>>>> On Mon, Jan 23, 2023 aat 11:12 AM Joe Witt <jo...@gmail.com>>
wrote:
br/>>>>>> Hello <
br/>>>>>> Here is a goodd picture of what the 1.20 RC looks like.
I've tagged several JIRAs today to ensure we get them in. A theme is really
around stabilizing nifi/nifi-registry integration as we're seeing
substantial uptick in usage and thus various community reported findings.
We'll get that quite smooth with these included.
br/>>>>>> https://issues<https://issues/>..
apache.org/jira/projects/NIFI/versions/12352581<http://apache.org/jira/projects/NIFI/versions/12352581>
br/>>>>>> Thanks <
br/>>>>>> On Mon, Jan 233, 2023 at 8:50 AM Joe Witt <
joe.witt@gmail.com<ma...@gmail.com>> wrote:
br/>>>>>>> Team, <
br/>>>>>>> I'm goiing through the outstanding JIRAs/PRs and flagging
which look like they should be 'must have' for 1.20 and then will work the
RC as soon as those land.
br/>>>>>>> Hopefullly have the RC up within a day or two but we'll
see how these land as some have review comments pending action.
br/>>>>>>> Thanks
br/>>>>>>> On Thu,, Jan 12, 2023 at 2:53 AM Isha Lamboo <
isha.lamboo@virtualsciences.nl<ma...@virtualsciences.nl>> wrote:
br/>>>>>>>>; Hi all,
br/>>>>>>>>; I would like to contribute to the migration tooling
(mostly testing I suppose) when that comes up.
br/>>>>>>>>; My team's largest client has a completely
template-based pipeline with external scripts replacing variable values
before deploying to target clusters, so we've already started looking at
this when the goals for 2.0 were discussed and approved. The migration to
flowdefinitions and parameters is quite complex and we've hit several
blockers when we tried to implement a direct translation.
br/>>>>>>>>; I expect that any time I spend helping to improve the
tooling will pay off handsomely for our clients.
br/>>>>>>>>; Regards,
br/>>>>>>>>; Isha
br/>>>>>>>>; -----Oorspronkelijk bericht-----
Van: Adam Taft <ad...@adamtaft.com>>
Verzonden: woensdag 11 januari 2023 23:42
Aan: dev@nifi.apache.org<ma...@nifi.apache.org>
Onderwerp: Re: [discuss] NiFi 1.20 and NiFi 2.0
br/>>>>>>>>; This is really insightful and spot on ...
br/>>>>>>>>; Kevin wrote:
Good migration tooling will take a while to develop and test, and
the
core contributors to that effort may not have sufficient variety
of
flows to evaluate when the migration tools are "done" for the
majority
of the community to have success upgrading to 2.x. A milestone
release
would
allow
us get more feedback on migration over a longer period than the
vote
window
of an RC candidate.
br/>>>>>>>>; It's exactly this case, that an early 2.0 release
might not have had time to fully work its way through existing production
deployments, that's concerning. The pace and voting of an "RC" is much too
short to get any quality feedback from users in the field.
br/>>>>>>>>; I think it's really smart to consider the "Milestone"
release approach here. We release 2.0.0-M1, 2.0.0-M2, ... waiting an
adequate amount of time for feedback. We can put these milestones on a
calendar, as needed, so that feedback is required some 'x' number of
weeks/months after each milestone.
br/>>>>>>>>; And to this end, I'd personally rather see us keep the
'main' branch current with the 1.x line _until_ we're ready and are
satisfied with the end goals of the 2.0 release objectives. When the
milestone releases have been completed and there's a comfort level with the
2.x line, it's at the point we'd isolate the 1.x line into its own branch
and switch main over to the 2.x line.
br/>>>>>>>>; This is an attractive way of:
a) continuing business-as-usual with the 1.x line
b) making headway on the 2.x release milestones
c) giving adequate time for feedback against the 2.0 milestones
coming from the field
br/>>>>>>>>; I don't mind the known-unknowns. But it's really the
unknown-unknowns that are going to drive a delay in the 2.0 release. I
think it's smart to be able to get some of the unknowns ironed out before
we finalize the 2.0 release ceremony. The milestone approach really helps
with that.
br/>>>>>>>>; /Adam
br/>>>>>>>>; On Wed, Jan 11, 2023 at 11:11 AM Kevin Doran <
kdoran@apache.org<ma...@apache.org>> wrote:
br/>>>>>>>>;> Sorry, Joe, I was not clear, and to be honest the two
thoughts are
somewhat unrelated in my mind too :)
br/>>>>>>>;>> I agree that good migration tooling is key.
Otherwise, we risk users
staying on 1.x or creating a schism of 1.x and 2.x users.
br/>>>>>>>;>> Good migration tooling will take a while to develop
and test, and the
core contributors to that effort may not have sufficient variety
of
flows to evaluate when the migration tools are "done" for the
majority
of the community to have success upgrading to 2.x. A milestone
release
would allow us get more feedback on migration over a longer period
than the vote window of an RC candidate.
br/>>>>>>>;>> Perhaps we could continue to release from the 1.x
line (including
minor releases with some features) until we are ready to drop the
"milestone"
qualifier from 2.0.0, and only then put 1.x into maintenance-only
status.
It would be the same proposal to move main to target 2.0.0-M1, but
relax restrictions for what can land on the 1.x branch and be open
to
a 1.21, 1.22, etc. if 2.0.0 work takes longer than anticipated.
For
example, maybe we would be open to landing new/backported
processors
on the 1.x branch, but not core framework features or API changes.
br/>>>>>>>;>> This might not be necessary, but I think it is fair
that saying "no
new features on 1.x" and also "no new features in 2.0.0" puts the
project in a rough place if 2.0.0 takes longer than a few months,
so
if we go that route, we need to commit to a quick release of 2.0.0
that most users can move to easily.
br/>>>>>>>;>> Thanks,
Kevin
br/>>>>>>>;>> On Jan 11, 2023 at 12:32:46, Joe Witt <
joe.witt@gmail.com<ma...@gmail.com>> wrote:
br/>>>>>>>;>>> Kevin,
br/>>>>>>;>>>> Yeah we can do whatever we want as far as
'releases' of 2.0 that are
prior
to us officially considering it 2.0/stable.
br/>>>>>>;>>>> That said - the migration tooling will be key to
provide as we need
to
make
the bridge as solid and stable as possible to help someone move
from
1.x
to
2.x. I dont know how related these two concepts (milestone
releases
and 1.x to 2.x ease really are).
br/>>>>>>;>>>> Thanks
br/>>>>>>;>>>> On Wed, Jan 11, 2023 at 10:27 AM Kevin Doran <
kdoran@apache.org<ma...@apache.org>> wrote:
br/>>>>>>;>>>> [hit the wrong keyboard shortcut, here is the rest
of my thoughts]
br/>>>>>>;>>>> br/>>>>>>>>>> On this pooint from David:
br/>>>>>>;>>>> br/>>>>>>>>>> We may neeed to have a longer
release candidate period, or more
incremental
br/>>>>>>;>>>>> fix releases
br/>>>>>>;>>>>> for the initial 2.0.0 release train, but I do not
expect delaying
a
2.0.0
br/>>>>>>;>>>>> release for new features, as that is not part of
the release goals.
br/>>>>>>;>>>>> br/>>>>>>>>>> br/>>>>>>>>>> br/>>>>>>>>>> Would
the community benefit from one or more milestone releases of
2.0,
to
br/>>>>>>;>>>> allow for a wider group to run / live on the
proposed 2.0 prior to
br/>>>>>>;>>>> releasing it as "stable"? I know we've never done
a milestone
release in
br/>>>>>>;>>>> the past, and I'm not sure what ASF guidance is on
the topic, but if
it
br/>>>>>>;>>>> could be beneficial we could look into that.
br/>>>>>>;>>>> br/>>>>>>>>>> Cheers, <
br/>>>>>>t;>>>> Kevin
br/>>>>>>;>>>> br/>>>>>>>>>> On Jan 11,, 2023 at 12:22:43, Kevin
Doran <kd...@apache.org>> wrote:
br/>>>>>>;>>>> br/>>>>>>>>>>> I thinnk this is a good, practical
discussion.
br/>>>>>>;>>>>> br/>>>>>>>>>> br/>>>>>>>>>>> On the one hand, we
can''t put off 2.x any longer as we really need
to
br/>>>>>>;>>>>> updated the minimum required Java to 11. Updating
main development
to
br/>>>>>>;>>>>> target 2.x feels like a good way drive progress
on that along with
the
br/>>>>>>;>>>>> other 2.0 goals.
br/>>>>>>;>>>>> br/>>>>>>>>>> br/>>>>>>>>>>> On the other hand,
the cconcerns are valid: moving all development
to
br/>>>>>>;>>>>> target 2.x puts the project at risk if we cannot
release 2.0.0 on
a
br/>>>>>>;>>>>> reasonable timeline. The restricted scope of 2.0
helps, but this
stated
br/>>>>>>;>>>>> release goal feels risky to me:
br/>>>>>>;>>>>> br/>>>>>>>>>> br/>>>>>>>>>>> Implement Migration
Toolls for Upgrading Flows
br/>>>>>>;>>>>> br/>>>>>>>>>> br/>>>>>>>>>>> br/>>>>>>>>>>
br/>>>>>>>>>>> - Implement automated migration where possible to remap
properties
and
br/>>>>>>;>>>>> features
br/>>>>>>;>>>>> - Implement migration tools for manual conversion
of XML
Templates
br/>>>>>>;>>>>> to JSON Flow Definitions
br/>>>>>>;>>>>> - Create documentation for manual steps necessary
where
br/>>>>>>;>>>>> programmatic migration cannot be implemented
br/>>>>>>;>>>>> - NiFi 2.0 should be capable of starting with
ghosted
components
br/>>>>>>;>>>>> for removed Processors or Controller Services
br/>>>>>>;>>>>> br/>>>>>>>>>> br/>>>>>>>>>>> br/>>>>>>>>>>
br/>>>>>>>>>>> Removing deprecated compponents should be fairly
straightforward
and
br/>>>>>>;>>>> quick,
br/>>>>>>;>>>>> but automating and documenting migration is a
large effort.
br/>>>>>>;>>>>> br/>>>>>>>>>> br/>>>>>>>>>>> On this po <
br/>>>>>>t;>>>>> br/>>>>>>>>>> br/>>>>>>>>>>> br/>>>>>>>>>>
br/>>>>>>>>>>> On Jan 10, 2023 at 09:322:31, Bryan Bende <bb...@gmail.com>>
wrote:
br/>>>>>>;>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> The plan as I
undersstand it is not to diverge and create separate
br/>>>>>>;>>>> feature
br/>>>>>>;>>>>>> development on the 1.x line, so I would expect
all PRs to
continue to
be
br/>>>>>>;>>>>>> submitted only to main. We would release 1.x as
needed with major
bug
br/>>>>>>;>>>>>> fixes
br/>>>>>>;>>>>>> or critical security updates, and these would be
cherry-picked
and/or
br/>>>>>>;>>>>>> backported as necessary, mostly without the need
for PRs, the
same as
we
br/>>>>>>;>>>>>> would do if we were bringing fixes from main
(1.20.0-SNAPSHOT)
back
to a
br/>>>>>>;>>>>>> maintenance line like (1.19.x). For precedent,
we followed this
same
br/>>>>>>;>>>>>> approach going from the 0.x line to 1.0.0 and
there wasn't any
major
br/>>>>>>;>>>>>> issue.
br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> On Tue, Jan 10,
20233 at 7:07 AM Otto Fowler
<ot...@gmail.com>>
br/>>>>>>;>>>>>> wrote:
br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> It was also
mentioneed in another thread that we need to have
agreement
br/>>>>>>;>>>> on
br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> our explicit
strateggy and support for 1.x going forward, did that
br/>>>>>>;>>>> happen?
br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> br/>>>>>>>>>>
br/>>>>>>>>>>>> FFrom: Otto Fowler <ot...@gmail.com>>
<ot...@gmail.com>>
br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> Reply: Otto
FFowler <ot...@gmail.com>>
<ot...@gmail.com>
br/>>>>>>;>>>> br/>>>>>>>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>>
Date: January 10, 20023 at 07:02:34
br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> To: dev@@
nifi.apache.org<http://nifi.apache.org/> <de...@nifi.apache.org>>
<de...@nifi.apache.org>>
br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> Subject: Re:
[[discuss] NiFi 1.20 and NiFi 2.0
br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> br/>>>>>>>>>>
br/>>>>>>>>>>>> There needs to be ann update to the contributing guide as
to how
to
br/>>>>>>;>>>> submit
br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> PRs to 1.x or 2.x
ettc.
br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> br/>>>>>>>>>>
br/>>>>>>>>>>>> FFrom: Joe Witt <jo...@apache.org>> <jo...@apache.org>>
br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> Reply: dev@@
nifi.apache.org<http://nifi.apache.org/> <de...@nifi.apache.org>>
<de...@nifi.apache.org>
br/>>>>>>;>>>> br/>>>>>>>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>>
Date: January 9, 20223 at 15:53:16
br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> To: dev@@
nifi.apache.org<http://nifi.apache.org/> <de...@nifi.apache.org>>
<de...@nifi.apache.org>>
br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> Subject:
[[discuss] NiFi 1.20 and NiFi 2.0
br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> br/>>>>>>>>>>
br/>>>>>>>>>>>> Team, <
br/>>>>>>t;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> br/>>>>>>>>>>
br/>>>>>>>>>>>> As David mentioned iin [1] following a successful NiFi 2.0
release
goal
br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> planning - we are
noow going to start moving the 'main' line to be
the
br/>>>>>>;>>>> NiFi
br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> 2.0 line which
will allow for the key work to take place. We will
also
br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> move niFFi 1.x to
its appropriate support line.
br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> br/>>>>>>>>>>
br/>>>>>>>>>>>> It is also the case that we have nearly 100 JIRAs on NiFi
1.20
and we
br/>>>>>>;>>>> have
br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> work in there
includding security items so it is time to make a
release.
br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> The intent then is
tto initiate 1.20 and immediate after that
change
br/>>>>>>;>>>> 'main'
br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> to 2.0. <
br/>>>>>>t;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> br/>>>>>>>>>>
br/>>>>>>>>>>>> Going forward then aall work on the 1.x line should be
focused on
br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> maintaining
existingg features, dependencies, and helping 1.x
users
br/>>>>>>;>>>> migrate
br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> to the 2.x line.
Othherwise, new feature work will happen on
'main' as
it
br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> normally does and
wiill come out in the NiFi 2.x release line.
br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> br/>>>>>>>>>>
br/>>>>>>>>>>>> Please flag key outsstanding items as we narrow down the
release
br/>>>>>>;>>>> candidate
br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> for NiFFi 1.20.
br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> br/>>>>>>>>>>
br/>>>>>>>>>>>> Thanks <
br/>>>>>>t;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> Joe <
br/>>>>>>t;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> br/>>>>>>>>>>
br/>>>>>>>>>>>> [[1]
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2
Flists.apache.org%2Fthread%2Fqo4vvdw46235y7vy2crcd6l4m11wl7jz&dat
a=05%7C01%7Cisha.lamboo%40virtualsciences.nl%7Ccbea974a2c1f479d48
9d08daf42521f1%7C21429da9e4ad45f99a6fcd126a64274b%7C0%7C0%7C63809
0737572228694%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjo
iV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=zSqOK9zZPqXxLuxwo0QcqKGEAc7aXjfnnm4i%2BQt2B98%3D&reserved=0
br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> br/>>>>>>>>>>
br/>>>>>>>>>>>> br/>>>>>>>>>> br/>>>>>>>>>> br/>>>>>>>>>> br/>>>>>>>>> br/>
<
Re: [discuss] NiFi 1.20 and NiFi 2.0
Posted by Otto Fowler <ot...@gmail.com>.
Thanks Mark,
Can you give an example of best practices cherry picking? This cherry
picked commit ( and any other re-work required to make it work ) will be a
new PR then? Or is that only if it doesn’t go in clean?
Contributor:
- Pull Request
Committer:
- Merge
- Cherry Pick to new PR branch off of support/nifi-1.x
- possible fixups
- push a new PR
Should the support/nifi-1.x PRs have a standard name format WRT the
original? Should the name be the same as the subject of the other, or
should it be the PR#?
From: Mark Payne <ma...@hotmail.com> <ma...@hotmail.com>
Reply: dev@nifi.apache.org <de...@nifi.apache.org> <de...@nifi.apache.org>
Date: February 10, 2023 at 11:51:28
To: dev@nifi.apache.org <de...@nifi.apache.org> <de...@nifi.apache.org>
Subject: Re: [discuss] NiFi 1.20 and NiFi 2.0
Thanks Joe!
So at this point, we now have the 1.x and the 2.x line diverging. As a
result, we will want to ensure that we keep the 1.x line up-to-date with
any bug fixes.
Trying to cherry-pick over just the necessary bug fixes for a release down
the road has the potential to be a nightmare, since some PRs may not apply
cleanly.
In order to avoid this: committers please ensure that for any bug fixes, if
you merge to the “main” branch that you also cherry-pick the commit to the
“support/nifi-1.x” branch.
If you’ve not dealt with support branches in the past, there’s one small
gotcha, as the branch name has a slash (/) in it. So to checkout the
support branch, you’ll use:
git checkout origin/support/nifi-1.x
And not just “git checkout support/nifi-1.x”
Then, you can create a local branch:
git checkout -b support/nifi-1.x
And after cherry-picking the necessary commits, you can push:
git push -u origin support/nifi-1.x
This will help to ensure that we keep a clean support branch so that we’re
able to make a release any time if the need arises, which will include all
of the necessary bug fixes.
Thanks
-Mark
> On Feb 9, 2023, at 5:43 PM, Joe Witt <jo...@gmail.com> wrote:
> br/>> Team, <
> br/>> Alrighty Apache NiFFi 1.20.0 is official!
> br/>> As a result our Apache NiFFi git branch 'main' is now officially
set up
> as our go forward line and is 2.0.0-SNAPSHOT as of now.
> br/>> To continue releases as needed for the 1.x linee such as
> 1.21.0-SNAPSHOT as would come next we now have 'support/nifi-1.x'. It
> is already setup for 1.21.0-SNAPSHOT versioning. We will need at
> least one such release on that line for migration tooling to help
> users move from that 1.x line to the 2.x line but depending on how
> long 2.0 takes to settle we may have more. Whatever it takes.
> br/>> Exciting times!!
> br/>> Thanks <
> Joe
> br/>> On Tue, FFeb 7, 2023 at 8:40 AM Joe Witt <jo...@gmail.com>
wrote:
>> br/>>> Team <
>> br/>>> As you commit to the main line now pleease start using whatever
the latest relevant feature release is on the 2.x line which right now
would be 2.0.0. If we find we need to do a 1.21 and so on we'll pull those
commits down as we would for any other mx release in the past but the
'main' line becomes 2.0.0 now.
>> br/>>> Thanks <
>> Joe
>> br/>>> On Mon, FFeb 6, 2023 at 10:44 AM Joe Witt <jo...@gmail.com>
wrote:
>>> br/>>>> Team, <
>>> br/>>>> I think we are there!! Going to kick out RC1 now.
>>> br/>>>> Thanks <
>>> br/>>>> On Mon, Jan 30, 2023 at 11:29 AM Joe Witt <jo...@gmail.com>
wrote:
>>>> br/>>>>> Team, <
>>>> br/>>>>> As you can see I've noot kicked off the RC yet. Many bug
fixes/dependency updates are happening. Ideally we'll wait until nar Maven
plugin goes and we're trying to fix some nifi registry/nifi interaction
issues as well. Still will get the RC out as soon as we can.
>>>> br/>>>>> Thanks <
>>>> br/>>>>> On Mon, Jan 23, 2023 aat 11:12 AM Joe Witt <jo...@gmail.com>
wrote:
>>>>> br/>>>>>> Hello <
>>>>> br/>>>>>> Here is a goodd picture of what the 1.20 RC looks like.
I've tagged several JIRAs today to ensure we get them in. A theme is really
around stabilizing nifi/nifi-registry integration as we're seeing
substantial uptick in usage and thus various community reported findings.
We'll get that quite smooth with these included.
>>>>> br/>>>>>> https://issues..
apache.org/jira/projects/NIFI/versions/12352581
>>>>> br/>>>>>> Thanks <
>>>>> br/>>>>>> On Mon, Jan 233, 2023 at 8:50 AM Joe Witt <
joe.witt@gmail.com> wrote:
>>>>>> br/>>>>>>> Team, <
>>>>>> br/>>>>>>> I'm goiing through the outstanding JIRAs/PRs and flagging
which look like they should be 'must have' for 1.20 and then will work the
RC as soon as those land.
>>>>>> br/>>>>>>> Hopefullly have the RC up within a day or two but we'll
see how these land as some have review comments pending action.
>>>>>> br/>>>>>>> Thanks
>>>>>> br/>>>>>>> On Thu,, Jan 12, 2023 at 2:53 AM Isha Lamboo <
isha.lamboo@virtualsciences.nl> wrote:
>>>>>>> br/>>>>>>>>; Hi all,
>>>>>>> br/>>>>>>>>; I would like to contribute to the migration tooling
(mostly testing I suppose) when that comes up.
>>>>>>> br/>>>>>>>>; My team's largest client has a completely
template-based pipeline with external scripts replacing variable values
before deploying to target clusters, so we've already started looking at
this when the goals for 2.0 were discussed and approved. The migration to
flowdefinitions and parameters is quite complex and we've hit several
blockers when we tried to implement a direct translation.
>>>>>>> br/>>>>>>>>; I expect that any time I spend helping to improve the
tooling will pay off handsomely for our clients.
>>>>>>> br/>>>>>>>>; Regards,
>>>>>>> br/>>>>>>>>; Isha
>>>>>>> br/>>>>>>>>; -----Oorspronkelijk bericht-----
>>>>>>> Van: Adam Taft <ad...@adamtaft.com>
>>>>>>> Verzonden: woensdag 11 januari 2023 23:42
>>>>>>> Aan: dev@nifi.apache.org
>>>>>>> Onderwerp: Re: [discuss] NiFi 1.20 and NiFi 2.0
>>>>>>> br/>>>>>>>>; This is really insightful and spot on ...
>>>>>>> br/>>>>>>>>; Kevin wrote:
>>>>>>>> Good migration tooling will take a while to develop and test, and
the
>>>>>>>> core contributors to that effort may not have sufficient variety
of
>>>>>>>> flows to evaluate when the migration tools are "done" for the
majority
>>>>>>>> of the community to have success upgrading to 2.x. A milestone
release
>>>>>>>> would
>>>>>>> allow
>>>>>>>> us get more feedback on migration over a longer period than the
vote
>>>>>>> window
>>>>>>>> of an RC candidate.
>>>>>>> br/>>>>>>>>; It's exactly this case, that an early 2.0 release
might not have had time to fully work its way through existing production
deployments, that's concerning. The pace and voting of an "RC" is much too
short to get any quality feedback from users in the field.
>>>>>>> br/>>>>>>>>; I think it's really smart to consider the "Milestone"
release approach here. We release 2.0.0-M1, 2.0.0-M2, ... waiting an
adequate amount of time for feedback. We can put these milestones on a
calendar, as needed, so that feedback is required some 'x' number of
weeks/months after each milestone.
>>>>>>> br/>>>>>>>>; And to this end, I'd personally rather see us keep the
'main' branch current with the 1.x line _until_ we're ready and are
satisfied with the end goals of the 2.0 release objectives. When the
milestone releases have been completed and there's a comfort level with the
2.x line, it's at the point we'd isolate the 1.x line into its own branch
and switch main over to the 2.x line.
>>>>>>> br/>>>>>>>>; This is an attractive way of:
>>>>>>> a) continuing business-as-usual with the 1.x line
>>>>>>> b) making headway on the 2.x release milestones
>>>>>>> c) giving adequate time for feedback against the 2.0 milestones
coming from the field
>>>>>>> br/>>>>>>>>; I don't mind the known-unknowns. But it's really the
unknown-unknowns that are going to drive a delay in the 2.0 release. I
think it's smart to be able to get some of the unknowns ironed out before
we finalize the 2.0 release ceremony. The milestone approach really helps
with that.
>>>>>>> br/>>>>>>>>; /Adam
>>>>>>> br/>>>>>>>>; On Wed, Jan 11, 2023 at 11:11 AM Kevin Doran <
kdoran@apache.org> wrote:
>>>>>>> br/>>>>>>>>;> Sorry, Joe, I was not clear, and to be honest the two
thoughts are
>>>>>>>> somewhat unrelated in my mind too :)
>>>>>>>> br/>>>>>>>;>> I agree that good migration tooling is key.
Otherwise, we risk users
>>>>>>>> staying on 1.x or creating a schism of 1.x and 2.x users.
>>>>>>>> br/>>>>>>>;>> Good migration tooling will take a while to develop
and test, and the
>>>>>>>> core contributors to that effort may not have sufficient variety
of
>>>>>>>> flows to evaluate when the migration tools are "done" for the
majority
>>>>>>>> of the community to have success upgrading to 2.x. A milestone
release
>>>>>>>> would allow us get more feedback on migration over a longer period
>>>>>>>> than the vote window of an RC candidate.
>>>>>>>> br/>>>>>>>;>> Perhaps we could continue to release from the 1.x
line (including
>>>>>>>> minor releases with some features) until we are ready to drop the
"milestone"
>>>>>>>> qualifier from 2.0.0, and only then put 1.x into maintenance-only
status.
>>>>>>>> It would be the same proposal to move main to target 2.0.0-M1, but
>>>>>>>> relax restrictions for what can land on the 1.x branch and be open
to
>>>>>>>> a 1.21, 1.22, etc. if 2.0.0 work takes longer than anticipated.
For
>>>>>>>> example, maybe we would be open to landing new/backported
processors
>>>>>>>> on the 1.x branch, but not core framework features or API changes.
>>>>>>>> br/>>>>>>>;>> This might not be necessary, but I think it is fair
that saying "no
>>>>>>>> new features on 1.x" and also "no new features in 2.0.0" puts the
>>>>>>>> project in a rough place if 2.0.0 takes longer than a few months,
so
>>>>>>>> if we go that route, we need to commit to a quick release of 2.0.0
>>>>>>>> that most users can move to easily.
>>>>>>>> br/>>>>>>>;>> Thanks,
>>>>>>>> Kevin
>>>>>>>> br/>>>>>>>;>> On Jan 11, 2023 at 12:32:46, Joe Witt <
joe.witt@gmail.com> wrote:
>>>>>>>> br/>>>>>>>;>>> Kevin,
>>>>>>>>> br/>>>>>>;>>>> Yeah we can do whatever we want as far as
'releases' of 2.0 that are
>>>>>>>> prior
>>>>>>>>> to us officially considering it 2.0/stable.
>>>>>>>>> br/>>>>>>;>>>> That said - the migration tooling will be key to
provide as we need
>>>>>>>>> to
>>>>>>>> make
>>>>>>>>> the bridge as solid and stable as possible to help someone move
from
>>>>>>>>> 1.x
>>>>>>>> to
>>>>>>>>> 2.x. I dont know how related these two concepts (milestone
releases
>>>>>>>>> and 1.x to 2.x ease really are).
>>>>>>>>> br/>>>>>>;>>>> Thanks
>>>>>>>>> br/>>>>>>;>>>> On Wed, Jan 11, 2023 at 10:27 AM Kevin Doran <
kdoran@apache.org> wrote:
>>>>>>>>> br/>>>>>>;>>>> [hit the wrong keyboard shortcut, here is the rest
of my thoughts]
>>>>>>>>> br/>>>>>>;>>>> br/>>>>>>>>>> On this pooint from David:
>>>>>>>>> br/>>>>>>;>>>> br/>>>>>>>>>> We may neeed to have a longer
release candidate period, or more
>>>>>>>> incremental
>>>>>>>>> br/>>>>>>;>>>>> fix releases
>>>>>>>>> br/>>>>>>;>>>>> for the initial 2.0.0 release train, but I do not
expect delaying
>>>>>>>>>> a
>>>>>>>> 2.0.0
>>>>>>>>> br/>>>>>>;>>>>> release for new features, as that is not part of
the release goals.
>>>>>>>>> br/>>>>>>;>>>>> br/>>>>>>>>>> br/>>>>>>>>>> br/>>>>>>>>>> Would
the community benefit from one or more milestone releases of
>>>>>>>>> 2.0,
>>>>>>>> to
>>>>>>>>> br/>>>>>>;>>>> allow for a wider group to run / live on the
proposed 2.0 prior to
>>>>>>>>> br/>>>>>>;>>>> releasing it as "stable"? I know we've never done
a milestone
>>>>>>>>> release in
>>>>>>>>> br/>>>>>>;>>>> the past, and I'm not sure what ASF guidance is on
the topic, but if
>>>>>>>>> it
>>>>>>>>> br/>>>>>>;>>>> could be beneficial we could look into that.
>>>>>>>>> br/>>>>>>;>>>> br/>>>>>>>>>> Cheers, <
>>>>>>>>> br/>>>>>>t;>>>> Kevin
>>>>>>>>> br/>>>>>>;>>>> br/>>>>>>>>>> On Jan 11,, 2023 at 12:22:43, Kevin
Doran <kd...@apache.org> wrote:
>>>>>>>>> br/>>>>>>;>>>> br/>>>>>>>>>>> I thinnk this is a good, practical
discussion.
>>>>>>>>> br/>>>>>>;>>>>> br/>>>>>>>>>> br/>>>>>>>>>>> On the one hand, we
can''t put off 2.x any longer as we really need
>>>>>>>>>> to
>>>>>>>>> br/>>>>>>;>>>>> updated the minimum required Java to 11. Updating
main development
>>>>>>>>>> to
>>>>>>>>> br/>>>>>>;>>>>> target 2.x feels like a good way drive progress
on that along with
>>>>>>>>>> the
>>>>>>>>> br/>>>>>>;>>>>> other 2.0 goals.
>>>>>>>>> br/>>>>>>;>>>>> br/>>>>>>>>>> br/>>>>>>>>>>> On the other hand,
the cconcerns are valid: moving all development
>>>>>>>>>> to
>>>>>>>>> br/>>>>>>;>>>>> target 2.x puts the project at risk if we cannot
release 2.0.0 on
>>>>>>>>>> a
>>>>>>>>> br/>>>>>>;>>>>> reasonable timeline. The restricted scope of 2.0
helps, but this
>>>>>>>>>> stated
>>>>>>>>> br/>>>>>>;>>>>> release goal feels risky to me:
>>>>>>>>> br/>>>>>>;>>>>> br/>>>>>>>>>> br/>>>>>>>>>>> Implement Migration
Toolls for Upgrading Flows
>>>>>>>>> br/>>>>>>;>>>>> br/>>>>>>>>>> br/>>>>>>>>>>> br/>>>>>>>>>>
br/>>>>>>>>>>> - Implement automated migration where possible to remap
>>>>>>>>>> properties
>>>>>>>> and
>>>>>>>>> br/>>>>>>;>>>>> features
>>>>>>>>> br/>>>>>>;>>>>> - Implement migration tools for manual conversion
of XML
>>>>>>>> Templates
>>>>>>>>> br/>>>>>>;>>>>> to JSON Flow Definitions
>>>>>>>>> br/>>>>>>;>>>>> - Create documentation for manual steps necessary
where
>>>>>>>>> br/>>>>>>;>>>>> programmatic migration cannot be implemented
>>>>>>>>> br/>>>>>>;>>>>> - NiFi 2.0 should be capable of starting with
ghosted
>>>>>>>>>> components
>>>>>>>>> br/>>>>>>;>>>>> for removed Processors or Controller Services
>>>>>>>>> br/>>>>>>;>>>>> br/>>>>>>>>>> br/>>>>>>>>>>> br/>>>>>>>>>>
br/>>>>>>>>>>> Removing deprecated compponents should be fairly
straightforward
>>>>>>>>>> and
>>>>>>>>> br/>>>>>>;>>>> quick,
>>>>>>>>> br/>>>>>>;>>>>> but automating and documenting migration is a
large effort.
>>>>>>>>> br/>>>>>>;>>>>> br/>>>>>>>>>> br/>>>>>>>>>>> On this po <
>>>>>>>>> br/>>>>>>t;>>>>> br/>>>>>>>>>> br/>>>>>>>>>>> br/>>>>>>>>>>
br/>>>>>>>>>>> On Jan 10, 2023 at 09:322:31, Bryan Bende <bb...@gmail.com>
wrote:
>>>>>>>>> br/>>>>>>;>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> The plan as I
undersstand it is not to diverge and create separate
>>>>>>>>> br/>>>>>>;>>>> feature
>>>>>>>>> br/>>>>>>;>>>>>> development on the 1.x line, so I would expect
all PRs to
>>>>>>>>>>> continue to
>>>>>>>> be
>>>>>>>>> br/>>>>>>;>>>>>> submitted only to main. We would release 1.x as
needed with major
>>>>>>>>>>> bug
>>>>>>>>> br/>>>>>>;>>>>>> fixes
>>>>>>>>> br/>>>>>>;>>>>>> or critical security updates, and these would be
cherry-picked
>>>>>>>>>>> and/or
>>>>>>>>> br/>>>>>>;>>>>>> backported as necessary, mostly without the need
for PRs, the
>>>>>>>>>>> same as
>>>>>>>> we
>>>>>>>>> br/>>>>>>;>>>>>> would do if we were bringing fixes from main
(1.20.0-SNAPSHOT)
>>>>>>>>>>> back
>>>>>>>> to a
>>>>>>>>> br/>>>>>>;>>>>>> maintenance line like (1.19.x). For precedent,
we followed this
>>>>>>>>>>> same
>>>>>>>>> br/>>>>>>;>>>>>> approach going from the 0.x line to 1.0.0 and
there wasn't any
>>>>>>>>>>> major
>>>>>>>>> br/>>>>>>;>>>>>> issue.
>>>>>>>>> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> On Tue, Jan 10,
20233 at 7:07 AM Otto Fowler
>>>>>>>>>>> <ot...@gmail.com>
>>>>>>>>> br/>>>>>>;>>>>>> wrote:
>>>>>>>>> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> It was also
mentioneed in another thread that we need to have
>>>>>>>> agreement
>>>>>>>>> br/>>>>>>;>>>> on
>>>>>>>>> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> our explicit
strateggy and support for 1.x going forward, did that
>>>>>>>>> br/>>>>>>;>>>> happen?
>>>>>>>>> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> br/>>>>>>>>>>
br/>>>>>>>>>>>> FFrom: Otto Fowler <ot...@gmail.com>
>>>>>>>>>>> <ot...@gmail.com>
>>>>>>>>> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> Reply: Otto
FFowler <ot...@gmail.com>
>>>>>>>>>>> <ottobackwards@gmail.com
>>>>>>>>> br/>>>>>>;>>>> br/>>>>>>>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>>
Date: January 10, 20023 at 07:02:34
>>>>>>>>> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> To: dev@@
nifi.apache.org <de...@nifi.apache.org>
>>>>>>>>>>> <de...@nifi.apache.org>
>>>>>>>>> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> Subject: Re:
[[discuss] NiFi 1.20 and NiFi 2.0
>>>>>>>>> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> br/>>>>>>>>>>
br/>>>>>>>>>>>> There needs to be ann update to the contributing guide as
to how
>>>>>>>>>>> to
>>>>>>>>> br/>>>>>>;>>>> submit
>>>>>>>>> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> PRs to 1.x or 2.x
ettc.
>>>>>>>>> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> br/>>>>>>>>>>
br/>>>>>>>>>>>> FFrom: Joe Witt <jo...@apache.org> <jo...@apache.org>
>>>>>>>>> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> Reply: dev@@
nifi.apache.org <de...@nifi.apache.org>
>>>>>>>>>>> <dev@nifi.apache.org
>>>>>>>>> br/>>>>>>;>>>> br/>>>>>>>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>>
Date: January 9, 20223 at 15:53:16
>>>>>>>>> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> To: dev@@
nifi.apache.org <de...@nifi.apache.org>
>>>>>>>>>>> <de...@nifi.apache.org>
>>>>>>>>> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> Subject:
[[discuss] NiFi 1.20 and NiFi 2.0
>>>>>>>>> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> br/>>>>>>>>>>
br/>>>>>>>>>>>> Team, <
>>>>>>>>> br/>>>>>>t;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> br/>>>>>>>>>>
br/>>>>>>>>>>>> As David mentioned iin [1] following a successful NiFi 2.0
release
>>>>>>>>>>> goal
>>>>>>>>> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> planning - we are
noow going to start moving the 'main' line to be
>>>>>>>>>>> the
>>>>>>>>> br/>>>>>>;>>>> NiFi
>>>>>>>>> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> 2.0 line which
will allow for the key work to take place. We will
>>>>>>>>>>> also
>>>>>>>>> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> move niFFi 1.x to
its appropriate support line.
>>>>>>>>> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> br/>>>>>>>>>>
br/>>>>>>>>>>>> It is also the case that we have nearly 100 JIRAs on NiFi
1.20
>>>>>>>>>>> and we
>>>>>>>>> br/>>>>>>;>>>> have
>>>>>>>>> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> work in there
includding security items so it is time to make a
>>>>>>>> release.
>>>>>>>>> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> The intent then is
tto initiate 1.20 and immediate after that
>>>>>>>>>>> change
>>>>>>>>> br/>>>>>>;>>>> 'main'
>>>>>>>>> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> to 2.0. <
>>>>>>>>> br/>>>>>>t;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> br/>>>>>>>>>>
br/>>>>>>>>>>>> Going forward then aall work on the 1.x line should be
focused on
>>>>>>>>> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> maintaining
existingg features, dependencies, and helping 1.x
>>>>>>>>>>> users
>>>>>>>>> br/>>>>>>;>>>> migrate
>>>>>>>>> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> to the 2.x line.
Othherwise, new feature work will happen on
>>>>>>>>>>> 'main' as
>>>>>>>> it
>>>>>>>>> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> normally does and
wiill come out in the NiFi 2.x release line.
>>>>>>>>> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> br/>>>>>>>>>>
br/>>>>>>>>>>>> Please flag key outsstanding items as we narrow down the
release
>>>>>>>>> br/>>>>>>;>>>> candidate
>>>>>>>>> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> for NiFFi 1.20.
>>>>>>>>> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> br/>>>>>>>>>>
br/>>>>>>>>>>>> Thanks <
>>>>>>>>> br/>>>>>>t;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> Joe <
>>>>>>>>> br/>>>>>>t;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> br/>>>>>>>>>>
br/>>>>>>>>>>>> [[1]
>>>>>>>>>>>
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2
>>>>>>>>>>> Flists.apache.org%2Fthread%2Fqo4vvdw46235y7vy2crcd6l4m11wl7jz&dat
>>>>>>>>>>> a=05%7C01%7Cisha.lamboo%40virtualsciences.nl%7Ccbea974a2c1f479d48
>>>>>>>>>>>
9d08daf42521f1%7C21429da9e4ad45f99a6fcd126a64274b%7C0%7C0%7C63809
>>>>>>>>>>>
0737572228694%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjo
>>>>>>>>>>>
iV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=zSqOK9zZPqXxLuxwo0QcqKGEAc7aXjfnnm4i%2BQt2B98%3D&reserved=0
>>>>>>>>> br/>>>>>>;>>>>>> br/>>>>>>>>>> br/>>>>>>>>>>>> br/>>>>>>>>>>
br/>>>>>>>>>>>> br/>>>>>>>>>> br/>>>>>>>>>> br/>>>>>>>>>> br/>>>>>>>>> br/>
<
Re: [discuss] NiFi 1.20 and NiFi 2.0
Posted by Mark Payne <ma...@hotmail.com>.
Thanks Joe!
So at this point, we now have the 1.x and the 2.x line diverging. As a result, we will want to ensure that we keep the 1.x line up-to-date with any bug fixes.
Trying to cherry-pick over just the necessary bug fixes for a release down the road has the potential to be a nightmare, since some PRs may not apply cleanly.
In order to avoid this: committers please ensure that for any bug fixes, if you merge to the “main” branch that you also cherry-pick the commit to the “support/nifi-1.x” branch.
If you’ve not dealt with support branches in the past, there’s one small gotcha, as the branch name has a slash (/) in it. So to checkout the support branch, you’ll use:
git checkout origin/support/nifi-1.x
And not just “git checkout support/nifi-1.x”
Then, you can create a local branch:
git checkout -b support/nifi-1.x
And after cherry-picking the necessary commits, you can push:
git push -u origin support/nifi-1.x
This will help to ensure that we keep a clean support branch so that we’re able to make a release any time if the need arises, which will include all of the necessary bug fixes.
Thanks
-Mark
> On Feb 9, 2023, at 5:43 PM, Joe Witt <jo...@gmail.com> wrote:
>
> Team,
>
> Alrighty Apache NiFi 1.20.0 is official!
>
> As a result our Apache NiFi git branch 'main' is now officially set up
> as our go forward line and is 2.0.0-SNAPSHOT as of now.
>
> To continue releases as needed for the 1.x line such as
> 1.21.0-SNAPSHOT as would come next we now have 'support/nifi-1.x'. It
> is already setup for 1.21.0-SNAPSHOT versioning. We will need at
> least one such release on that line for migration tooling to help
> users move from that 1.x line to the 2.x line but depending on how
> long 2.0 takes to settle we may have more. Whatever it takes.
>
> Exciting times!
>
> Thanks
> Joe
>
> On Tue, Feb 7, 2023 at 8:40 AM Joe Witt <jo...@gmail.com> wrote:
>>
>> Team
>>
>> As you commit to the main line now please start using whatever the latest relevant feature release is on the 2.x line which right now would be 2.0.0. If we find we need to do a 1.21 and so on we'll pull those commits down as we would for any other mx release in the past but the 'main' line becomes 2.0.0 now.
>>
>> Thanks
>> Joe
>>
>> On Mon, Feb 6, 2023 at 10:44 AM Joe Witt <jo...@gmail.com> wrote:
>>>
>>> Team,
>>>
>>> I think we are there! Going to kick out RC1 now.
>>>
>>> Thanks
>>>
>>> On Mon, Jan 30, 2023 at 11:29 AM Joe Witt <jo...@gmail.com> wrote:
>>>>
>>>> Team,
>>>>
>>>> As you can see I've not kicked off the RC yet. Many bug fixes/dependency updates are happening. Ideally we'll wait until nar Maven plugin goes and we're trying to fix some nifi registry/nifi interaction issues as well. Still will get the RC out as soon as we can.
>>>>
>>>> Thanks
>>>>
>>>> On Mon, Jan 23, 2023 at 11:12 AM Joe Witt <jo...@gmail.com> wrote:
>>>>>
>>>>> Hello
>>>>>
>>>>> Here is a good picture of what the 1.20 RC looks like. I've tagged several JIRAs today to ensure we get them in. A theme is really around stabilizing nifi/nifi-registry integration as we're seeing substantial uptick in usage and thus various community reported findings. We'll get that quite smooth with these included.
>>>>>
>>>>> https://issues.apache.org/jira/projects/NIFI/versions/12352581
>>>>>
>>>>> Thanks
>>>>>
>>>>> On Mon, Jan 23, 2023 at 8:50 AM Joe Witt <jo...@gmail.com> wrote:
>>>>>>
>>>>>> Team,
>>>>>>
>>>>>> I'm going through the outstanding JIRAs/PRs and flagging which look like they should be 'must have' for 1.20 and then will work the RC as soon as those land.
>>>>>>
>>>>>> Hopefully have the RC up within a day or two but we'll see how these land as some have review comments pending action.
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>> On Thu, Jan 12, 2023 at 2:53 AM Isha Lamboo <is...@virtualsciences.nl> wrote:
>>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> I would like to contribute to the migration tooling (mostly testing I suppose) when that comes up.
>>>>>>>
>>>>>>> My team's largest client has a completely template-based pipeline with external scripts replacing variable values before deploying to target clusters, so we've already started looking at this when the goals for 2.0 were discussed and approved. The migration to flowdefinitions and parameters is quite complex and we've hit several blockers when we tried to implement a direct translation.
>>>>>>>
>>>>>>> I expect that any time I spend helping to improve the tooling will pay off handsomely for our clients.
>>>>>>>
>>>>>>> Regards,
>>>>>>>
>>>>>>> Isha
>>>>>>>
>>>>>>> -----Oorspronkelijk bericht-----
>>>>>>> Van: Adam Taft <ad...@adamtaft.com>
>>>>>>> Verzonden: woensdag 11 januari 2023 23:42
>>>>>>> Aan: dev@nifi.apache.org
>>>>>>> Onderwerp: Re: [discuss] NiFi 1.20 and NiFi 2.0
>>>>>>>
>>>>>>> This is really insightful and spot on ...
>>>>>>>
>>>>>>> Kevin wrote:
>>>>>>>> Good migration tooling will take a while to develop and test, and the
>>>>>>>> core contributors to that effort may not have sufficient variety of
>>>>>>>> flows to evaluate when the migration tools are "done" for the majority
>>>>>>>> of the community to have success upgrading to 2.x. A milestone release
>>>>>>>> would
>>>>>>> allow
>>>>>>>> us get more feedback on migration over a longer period than the vote
>>>>>>> window
>>>>>>>> of an RC candidate.
>>>>>>>
>>>>>>> It's exactly this case, that an early 2.0 release might not have had time to fully work its way through existing production deployments, that's concerning. The pace and voting of an "RC" is much too short to get any quality feedback from users in the field.
>>>>>>>
>>>>>>> I think it's really smart to consider the "Milestone" release approach here. We release 2.0.0-M1, 2.0.0-M2, ... waiting an adequate amount of time for feedback. We can put these milestones on a calendar, as needed, so that feedback is required some 'x' number of weeks/months after each milestone.
>>>>>>>
>>>>>>> And to this end, I'd personally rather see us keep the 'main' branch current with the 1.x line _until_ we're ready and are satisfied with the end goals of the 2.0 release objectives. When the milestone releases have been completed and there's a comfort level with the 2.x line, it's at the point we'd isolate the 1.x line into its own branch and switch main over to the 2.x line.
>>>>>>>
>>>>>>> This is an attractive way of:
>>>>>>> a) continuing business-as-usual with the 1.x line
>>>>>>> b) making headway on the 2.x release milestones
>>>>>>> c) giving adequate time for feedback against the 2.0 milestones coming from the field
>>>>>>>
>>>>>>> I don't mind the known-unknowns. But it's really the unknown-unknowns that are going to drive a delay in the 2.0 release. I think it's smart to be able to get some of the unknowns ironed out before we finalize the 2.0 release ceremony. The milestone approach really helps with that.
>>>>>>>
>>>>>>> /Adam
>>>>>>>
>>>>>>> On Wed, Jan 11, 2023 at 11:11 AM Kevin Doran <kd...@apache.org> wrote:
>>>>>>>
>>>>>>>> Sorry, Joe, I was not clear, and to be honest the two thoughts are
>>>>>>>> somewhat unrelated in my mind too :)
>>>>>>>>
>>>>>>>> I agree that good migration tooling is key. Otherwise, we risk users
>>>>>>>> staying on 1.x or creating a schism of 1.x and 2.x users.
>>>>>>>>
>>>>>>>> Good migration tooling will take a while to develop and test, and the
>>>>>>>> core contributors to that effort may not have sufficient variety of
>>>>>>>> flows to evaluate when the migration tools are "done" for the majority
>>>>>>>> of the community to have success upgrading to 2.x. A milestone release
>>>>>>>> would allow us get more feedback on migration over a longer period
>>>>>>>> than the vote window of an RC candidate.
>>>>>>>>
>>>>>>>> Perhaps we could continue to release from the 1.x line (including
>>>>>>>> minor releases with some features) until we are ready to drop the "milestone"
>>>>>>>> qualifier from 2.0.0, and only then put 1.x into maintenance-only status.
>>>>>>>> It would be the same proposal to move main to target 2.0.0-M1, but
>>>>>>>> relax restrictions for what can land on the 1.x branch and be open to
>>>>>>>> a 1.21, 1.22, etc. if 2.0.0 work takes longer than anticipated. For
>>>>>>>> example, maybe we would be open to landing new/backported processors
>>>>>>>> on the 1.x branch, but not core framework features or API changes.
>>>>>>>>
>>>>>>>> This might not be necessary, but I think it is fair that saying "no
>>>>>>>> new features on 1.x" and also "no new features in 2.0.0" puts the
>>>>>>>> project in a rough place if 2.0.0 takes longer than a few months, so
>>>>>>>> if we go that route, we need to commit to a quick release of 2.0.0
>>>>>>>> that most users can move to easily.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Kevin
>>>>>>>>
>>>>>>>> On Jan 11, 2023 at 12:32:46, Joe Witt <jo...@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Kevin,
>>>>>>>>>
>>>>>>>>> Yeah we can do whatever we want as far as 'releases' of 2.0 that are
>>>>>>>> prior
>>>>>>>>> to us officially considering it 2.0/stable.
>>>>>>>>>
>>>>>>>>> That said - the migration tooling will be key to provide as we need
>>>>>>>>> to
>>>>>>>> make
>>>>>>>>> the bridge as solid and stable as possible to help someone move from
>>>>>>>>> 1.x
>>>>>>>> to
>>>>>>>>> 2.x. I dont know how related these two concepts (milestone releases
>>>>>>>>> and 1.x to 2.x ease really are).
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>>
>>>>>>>>> On Wed, Jan 11, 2023 at 10:27 AM Kevin Doran <kd...@apache.org> wrote:
>>>>>>>>>
>>>>>>>>> [hit the wrong keyboard shortcut, here is the rest of my thoughts]
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On this point from David:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> We may need to have a longer release candidate period, or more
>>>>>>>> incremental
>>>>>>>>>
>>>>>>>>>> fix releases
>>>>>>>>>
>>>>>>>>>> for the initial 2.0.0 release train, but I do not expect delaying
>>>>>>>>>> a
>>>>>>>> 2.0.0
>>>>>>>>>
>>>>>>>>>> release for new features, as that is not part of the release goals.
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Would the community benefit from one or more milestone releases of
>>>>>>>>> 2.0,
>>>>>>>> to
>>>>>>>>>
>>>>>>>>> allow for a wider group to run / live on the proposed 2.0 prior to
>>>>>>>>>
>>>>>>>>> releasing it as "stable"? I know we've never done a milestone
>>>>>>>>> release in
>>>>>>>>>
>>>>>>>>> the past, and I'm not sure what ASF guidance is on the topic, but if
>>>>>>>>> it
>>>>>>>>>
>>>>>>>>> could be beneficial we could look into that.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>>
>>>>>>>>> Kevin
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Jan 11, 2023 at 12:22:43, Kevin Doran <kd...@apache.org> wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> I think this is a good, practical discussion.
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On the one hand, we can't put off 2.x any longer as we really need
>>>>>>>>>> to
>>>>>>>>>
>>>>>>>>>> updated the minimum required Java to 11. Updating main development
>>>>>>>>>> to
>>>>>>>>>
>>>>>>>>>> target 2.x feels like a good way drive progress on that along with
>>>>>>>>>> the
>>>>>>>>>
>>>>>>>>>> other 2.0 goals.
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On the other hand, the concerns are valid: moving all development
>>>>>>>>>> to
>>>>>>>>>
>>>>>>>>>> target 2.x puts the project at risk if we cannot release 2.0.0 on
>>>>>>>>>> a
>>>>>>>>>
>>>>>>>>>> reasonable timeline. The restricted scope of 2.0 helps, but this
>>>>>>>>>> stated
>>>>>>>>>
>>>>>>>>>> release goal feels risky to me:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Implement Migration Tools for Upgrading Flows
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> - Implement automated migration where possible to remap
>>>>>>>>>> properties
>>>>>>>> and
>>>>>>>>>
>>>>>>>>>> features
>>>>>>>>>
>>>>>>>>>> - Implement migration tools for manual conversion of XML
>>>>>>>> Templates
>>>>>>>>>
>>>>>>>>>> to JSON Flow Definitions
>>>>>>>>>
>>>>>>>>>> - Create documentation for manual steps necessary where
>>>>>>>>>
>>>>>>>>>> programmatic migration cannot be implemented
>>>>>>>>>
>>>>>>>>>> - NiFi 2.0 should be capable of starting with ghosted
>>>>>>>>>> components
>>>>>>>>>
>>>>>>>>>> for removed Processors or Controller Services
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Removing deprecated components should be fairly straightforward
>>>>>>>>>> and
>>>>>>>>>
>>>>>>>>> quick,
>>>>>>>>>
>>>>>>>>>> but automating and documenting migration is a large effort.
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On this po
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On Jan 10, 2023 at 09:32:31, Bryan Bende <bb...@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> The plan as I understand it is not to diverge and create separate
>>>>>>>>>
>>>>>>>>> feature
>>>>>>>>>
>>>>>>>>>>> development on the 1.x line, so I would expect all PRs to
>>>>>>>>>>> continue to
>>>>>>>> be
>>>>>>>>>
>>>>>>>>>>> submitted only to main. We would release 1.x as needed with major
>>>>>>>>>>> bug
>>>>>>>>>
>>>>>>>>>>> fixes
>>>>>>>>>
>>>>>>>>>>> or critical security updates, and these would be cherry-picked
>>>>>>>>>>> and/or
>>>>>>>>>
>>>>>>>>>>> backported as necessary, mostly without the need for PRs, the
>>>>>>>>>>> same as
>>>>>>>> we
>>>>>>>>>
>>>>>>>>>>> would do if we were bringing fixes from main (1.20.0-SNAPSHOT)
>>>>>>>>>>> back
>>>>>>>> to a
>>>>>>>>>
>>>>>>>>>>> maintenance line like (1.19.x). For precedent, we followed this
>>>>>>>>>>> same
>>>>>>>>>
>>>>>>>>>>> approach going from the 0.x line to 1.0.0 and there wasn't any
>>>>>>>>>>> major
>>>>>>>>>
>>>>>>>>>>> issue.
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> On Tue, Jan 10, 2023 at 7:07 AM Otto Fowler
>>>>>>>>>>> <ot...@gmail.com>
>>>>>>>>>
>>>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> It was also mentioned in another thread that we need to have
>>>>>>>> agreement
>>>>>>>>>
>>>>>>>>> on
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> our explicit strategy and support for 1.x going forward, did that
>>>>>>>>>
>>>>>>>>> happen?
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> From: Otto Fowler <ot...@gmail.com>
>>>>>>>>>>> <ot...@gmail.com>
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> Reply: Otto Fowler <ot...@gmail.com>
>>>>>>>>>>> <ottobackwards@gmail.com
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> Date: January 10, 2023 at 07:02:34
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> To: dev@nifi.apache.org <de...@nifi.apache.org>
>>>>>>>>>>> <de...@nifi.apache.org>
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> Subject: Re: [discuss] NiFi 1.20 and NiFi 2.0
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> There needs to be an update to the contributing guide as to how
>>>>>>>>>>> to
>>>>>>>>>
>>>>>>>>> submit
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> PRs to 1.x or 2.x etc.
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> From: Joe Witt <jo...@apache.org> <jo...@apache.org>
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> Reply: dev@nifi.apache.org <de...@nifi.apache.org>
>>>>>>>>>>> <dev@nifi.apache.org
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> Date: January 9, 2023 at 15:53:16
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> To: dev@nifi.apache.org <de...@nifi.apache.org>
>>>>>>>>>>> <de...@nifi.apache.org>
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> Subject: [discuss] NiFi 1.20 and NiFi 2.0
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> Team,
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> As David mentioned in [1] following a successful NiFi 2.0 release
>>>>>>>>>>> goal
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> planning - we are now going to start moving the 'main' line to be
>>>>>>>>>>> the
>>>>>>>>>
>>>>>>>>> NiFi
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> 2.0 line which will allow for the key work to take place. We will
>>>>>>>>>>> also
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> move niFi 1.x to its appropriate support line.
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> It is also the case that we have nearly 100 JIRAs on NiFi 1.20
>>>>>>>>>>> and we
>>>>>>>>>
>>>>>>>>> have
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> work in there including security items so it is time to make a
>>>>>>>> release.
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> The intent then is to initiate 1.20 and immediate after that
>>>>>>>>>>> change
>>>>>>>>>
>>>>>>>>> 'main'
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> to 2.0.
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> Going forward then all work on the 1.x line should be focused on
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> maintaining existing features, dependencies, and helping 1.x
>>>>>>>>>>> users
>>>>>>>>>
>>>>>>>>> migrate
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> to the 2.x line. Otherwise, new feature work will happen on
>>>>>>>>>>> 'main' as
>>>>>>>> it
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> normally does and will come out in the NiFi 2.x release line.
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> Please flag key outstanding items as we narrow down the release
>>>>>>>>>
>>>>>>>>> candidate
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> for NiFi 1.20.
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> Thanks
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> Joe
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> [1]
>>>>>>>>>>> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2
>>>>>>>>>>> Flists.apache.org%2Fthread%2Fqo4vvdw46235y7vy2crcd6l4m11wl7jz&dat
>>>>>>>>>>> a=05%7C01%7Cisha.lamboo%40virtualsciences.nl%7Ccbea974a2c1f479d48
>>>>>>>>>>> 9d08daf42521f1%7C21429da9e4ad45f99a6fcd126a64274b%7C0%7C0%7C63809
>>>>>>>>>>> 0737572228694%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjo
>>>>>>>>>>> iV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=zSqOK9zZPqXxLuxwo0QcqKGEAc7aXjfnnm4i%2BQt2B98%3D&reserved=0
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
Re: [discuss] NiFi 1.20 and NiFi 2.0
Posted by Joe Witt <jo...@gmail.com>.
Team,
Alrighty Apache NiFi 1.20.0 is official!
As a result our Apache NiFi git branch 'main' is now officially set up
as our go forward line and is 2.0.0-SNAPSHOT as of now.
To continue releases as needed for the 1.x line such as
1.21.0-SNAPSHOT as would come next we now have 'support/nifi-1.x'. It
is already setup for 1.21.0-SNAPSHOT versioning. We will need at
least one such release on that line for migration tooling to help
users move from that 1.x line to the 2.x line but depending on how
long 2.0 takes to settle we may have more. Whatever it takes.
Exciting times!
Thanks
Joe
On Tue, Feb 7, 2023 at 8:40 AM Joe Witt <jo...@gmail.com> wrote:
>
> Team
>
> As you commit to the main line now please start using whatever the latest relevant feature release is on the 2.x line which right now would be 2.0.0. If we find we need to do a 1.21 and so on we'll pull those commits down as we would for any other mx release in the past but the 'main' line becomes 2.0.0 now.
>
> Thanks
> Joe
>
> On Mon, Feb 6, 2023 at 10:44 AM Joe Witt <jo...@gmail.com> wrote:
>>
>> Team,
>>
>> I think we are there! Going to kick out RC1 now.
>>
>> Thanks
>>
>> On Mon, Jan 30, 2023 at 11:29 AM Joe Witt <jo...@gmail.com> wrote:
>>>
>>> Team,
>>>
>>> As you can see I've not kicked off the RC yet. Many bug fixes/dependency updates are happening. Ideally we'll wait until nar Maven plugin goes and we're trying to fix some nifi registry/nifi interaction issues as well. Still will get the RC out as soon as we can.
>>>
>>> Thanks
>>>
>>> On Mon, Jan 23, 2023 at 11:12 AM Joe Witt <jo...@gmail.com> wrote:
>>>>
>>>> Hello
>>>>
>>>> Here is a good picture of what the 1.20 RC looks like. I've tagged several JIRAs today to ensure we get them in. A theme is really around stabilizing nifi/nifi-registry integration as we're seeing substantial uptick in usage and thus various community reported findings. We'll get that quite smooth with these included.
>>>>
>>>> https://issues.apache.org/jira/projects/NIFI/versions/12352581
>>>>
>>>> Thanks
>>>>
>>>> On Mon, Jan 23, 2023 at 8:50 AM Joe Witt <jo...@gmail.com> wrote:
>>>>>
>>>>> Team,
>>>>>
>>>>> I'm going through the outstanding JIRAs/PRs and flagging which look like they should be 'must have' for 1.20 and then will work the RC as soon as those land.
>>>>>
>>>>> Hopefully have the RC up within a day or two but we'll see how these land as some have review comments pending action.
>>>>>
>>>>> Thanks
>>>>>
>>>>> On Thu, Jan 12, 2023 at 2:53 AM Isha Lamboo <is...@virtualsciences.nl> wrote:
>>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> I would like to contribute to the migration tooling (mostly testing I suppose) when that comes up.
>>>>>>
>>>>>> My team's largest client has a completely template-based pipeline with external scripts replacing variable values before deploying to target clusters, so we've already started looking at this when the goals for 2.0 were discussed and approved. The migration to flowdefinitions and parameters is quite complex and we've hit several blockers when we tried to implement a direct translation.
>>>>>>
>>>>>> I expect that any time I spend helping to improve the tooling will pay off handsomely for our clients.
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> Isha
>>>>>>
>>>>>> -----Oorspronkelijk bericht-----
>>>>>> Van: Adam Taft <ad...@adamtaft.com>
>>>>>> Verzonden: woensdag 11 januari 2023 23:42
>>>>>> Aan: dev@nifi.apache.org
>>>>>> Onderwerp: Re: [discuss] NiFi 1.20 and NiFi 2.0
>>>>>>
>>>>>> This is really insightful and spot on ...
>>>>>>
>>>>>> Kevin wrote:
>>>>>> > Good migration tooling will take a while to develop and test, and the
>>>>>> > core contributors to that effort may not have sufficient variety of
>>>>>> > flows to evaluate when the migration tools are "done" for the majority
>>>>>> > of the community to have success upgrading to 2.x. A milestone release
>>>>>> > would
>>>>>> allow
>>>>>> > us get more feedback on migration over a longer period than the vote
>>>>>> window
>>>>>> > of an RC candidate.
>>>>>>
>>>>>> It's exactly this case, that an early 2.0 release might not have had time to fully work its way through existing production deployments, that's concerning. The pace and voting of an "RC" is much too short to get any quality feedback from users in the field.
>>>>>>
>>>>>> I think it's really smart to consider the "Milestone" release approach here. We release 2.0.0-M1, 2.0.0-M2, ... waiting an adequate amount of time for feedback. We can put these milestones on a calendar, as needed, so that feedback is required some 'x' number of weeks/months after each milestone.
>>>>>>
>>>>>> And to this end, I'd personally rather see us keep the 'main' branch current with the 1.x line _until_ we're ready and are satisfied with the end goals of the 2.0 release objectives. When the milestone releases have been completed and there's a comfort level with the 2.x line, it's at the point we'd isolate the 1.x line into its own branch and switch main over to the 2.x line.
>>>>>>
>>>>>> This is an attractive way of:
>>>>>> a) continuing business-as-usual with the 1.x line
>>>>>> b) making headway on the 2.x release milestones
>>>>>> c) giving adequate time for feedback against the 2.0 milestones coming from the field
>>>>>>
>>>>>> I don't mind the known-unknowns. But it's really the unknown-unknowns that are going to drive a delay in the 2.0 release. I think it's smart to be able to get some of the unknowns ironed out before we finalize the 2.0 release ceremony. The milestone approach really helps with that.
>>>>>>
>>>>>> /Adam
>>>>>>
>>>>>> On Wed, Jan 11, 2023 at 11:11 AM Kevin Doran <kd...@apache.org> wrote:
>>>>>>
>>>>>> > Sorry, Joe, I was not clear, and to be honest the two thoughts are
>>>>>> > somewhat unrelated in my mind too :)
>>>>>> >
>>>>>> > I agree that good migration tooling is key. Otherwise, we risk users
>>>>>> > staying on 1.x or creating a schism of 1.x and 2.x users.
>>>>>> >
>>>>>> > Good migration tooling will take a while to develop and test, and the
>>>>>> > core contributors to that effort may not have sufficient variety of
>>>>>> > flows to evaluate when the migration tools are "done" for the majority
>>>>>> > of the community to have success upgrading to 2.x. A milestone release
>>>>>> > would allow us get more feedback on migration over a longer period
>>>>>> > than the vote window of an RC candidate.
>>>>>> >
>>>>>> > Perhaps we could continue to release from the 1.x line (including
>>>>>> > minor releases with some features) until we are ready to drop the "milestone"
>>>>>> > qualifier from 2.0.0, and only then put 1.x into maintenance-only status.
>>>>>> > It would be the same proposal to move main to target 2.0.0-M1, but
>>>>>> > relax restrictions for what can land on the 1.x branch and be open to
>>>>>> > a 1.21, 1.22, etc. if 2.0.0 work takes longer than anticipated. For
>>>>>> > example, maybe we would be open to landing new/backported processors
>>>>>> > on the 1.x branch, but not core framework features or API changes.
>>>>>> >
>>>>>> > This might not be necessary, but I think it is fair that saying "no
>>>>>> > new features on 1.x" and also "no new features in 2.0.0" puts the
>>>>>> > project in a rough place if 2.0.0 takes longer than a few months, so
>>>>>> > if we go that route, we need to commit to a quick release of 2.0.0
>>>>>> > that most users can move to easily.
>>>>>> >
>>>>>> > Thanks,
>>>>>> > Kevin
>>>>>> >
>>>>>> > On Jan 11, 2023 at 12:32:46, Joe Witt <jo...@gmail.com> wrote:
>>>>>> >
>>>>>> > > Kevin,
>>>>>> > >
>>>>>> > > Yeah we can do whatever we want as far as 'releases' of 2.0 that are
>>>>>> > prior
>>>>>> > > to us officially considering it 2.0/stable.
>>>>>> > >
>>>>>> > > That said - the migration tooling will be key to provide as we need
>>>>>> > > to
>>>>>> > make
>>>>>> > > the bridge as solid and stable as possible to help someone move from
>>>>>> > > 1.x
>>>>>> > to
>>>>>> > > 2.x. I dont know how related these two concepts (milestone releases
>>>>>> > > and 1.x to 2.x ease really are).
>>>>>> > >
>>>>>> > > Thanks
>>>>>> > >
>>>>>> > > On Wed, Jan 11, 2023 at 10:27 AM Kevin Doran <kd...@apache.org> wrote:
>>>>>> > >
>>>>>> > > [hit the wrong keyboard shortcut, here is the rest of my thoughts]
>>>>>> > >
>>>>>> > >
>>>>>> > > On this point from David:
>>>>>> > >
>>>>>> > >
>>>>>> > > We may need to have a longer release candidate period, or more
>>>>>> > incremental
>>>>>> > >
>>>>>> > > > fix releases
>>>>>> > >
>>>>>> > > > for the initial 2.0.0 release train, but I do not expect delaying
>>>>>> > > > a
>>>>>> > 2.0.0
>>>>>> > >
>>>>>> > > > release for new features, as that is not part of the release goals.
>>>>>> > >
>>>>>> > > >
>>>>>> > >
>>>>>> > >
>>>>>> > > Would the community benefit from one or more milestone releases of
>>>>>> > > 2.0,
>>>>>> > to
>>>>>> > >
>>>>>> > > allow for a wider group to run / live on the proposed 2.0 prior to
>>>>>> > >
>>>>>> > > releasing it as "stable"? I know we've never done a milestone
>>>>>> > > release in
>>>>>> > >
>>>>>> > > the past, and I'm not sure what ASF guidance is on the topic, but if
>>>>>> > > it
>>>>>> > >
>>>>>> > > could be beneficial we could look into that.
>>>>>> > >
>>>>>> > >
>>>>>> > > Cheers,
>>>>>> > >
>>>>>> > > Kevin
>>>>>> > >
>>>>>> > >
>>>>>> > > On Jan 11, 2023 at 12:22:43, Kevin Doran <kd...@apache.org> wrote:
>>>>>> > >
>>>>>> > >
>>>>>> > > > I think this is a good, practical discussion.
>>>>>> > >
>>>>>> > > >
>>>>>> > >
>>>>>> > > > On the one hand, we can't put off 2.x any longer as we really need
>>>>>> > > > to
>>>>>> > >
>>>>>> > > > updated the minimum required Java to 11. Updating main development
>>>>>> > > > to
>>>>>> > >
>>>>>> > > > target 2.x feels like a good way drive progress on that along with
>>>>>> > > > the
>>>>>> > >
>>>>>> > > > other 2.0 goals.
>>>>>> > >
>>>>>> > > >
>>>>>> > >
>>>>>> > > > On the other hand, the concerns are valid: moving all development
>>>>>> > > > to
>>>>>> > >
>>>>>> > > > target 2.x puts the project at risk if we cannot release 2.0.0 on
>>>>>> > > > a
>>>>>> > >
>>>>>> > > > reasonable timeline. The restricted scope of 2.0 helps, but this
>>>>>> > > > stated
>>>>>> > >
>>>>>> > > > release goal feels risky to me:
>>>>>> > >
>>>>>> > > >
>>>>>> > >
>>>>>> > > > Implement Migration Tools for Upgrading Flows
>>>>>> > >
>>>>>> > > >
>>>>>> > >
>>>>>> > > >
>>>>>> > >
>>>>>> > > > - Implement automated migration where possible to remap
>>>>>> > > > properties
>>>>>> > and
>>>>>> > >
>>>>>> > > > features
>>>>>> > >
>>>>>> > > > - Implement migration tools for manual conversion of XML
>>>>>> > Templates
>>>>>> > >
>>>>>> > > > to JSON Flow Definitions
>>>>>> > >
>>>>>> > > > - Create documentation for manual steps necessary where
>>>>>> > >
>>>>>> > > > programmatic migration cannot be implemented
>>>>>> > >
>>>>>> > > > - NiFi 2.0 should be capable of starting with ghosted
>>>>>> > > > components
>>>>>> > >
>>>>>> > > > for removed Processors or Controller Services
>>>>>> > >
>>>>>> > > >
>>>>>> > >
>>>>>> > > >
>>>>>> > >
>>>>>> > > > Removing deprecated components should be fairly straightforward
>>>>>> > > > and
>>>>>> > >
>>>>>> > > quick,
>>>>>> > >
>>>>>> > > > but automating and documenting migration is a large effort.
>>>>>> > >
>>>>>> > > >
>>>>>> > >
>>>>>> > > > On this po
>>>>>> > >
>>>>>> > > >
>>>>>> > >
>>>>>> > > >
>>>>>> > >
>>>>>> > > > On Jan 10, 2023 at 09:32:31, Bryan Bende <bb...@gmail.com> wrote:
>>>>>> > >
>>>>>> > > >
>>>>>> > >
>>>>>> > > >> The plan as I understand it is not to diverge and create separate
>>>>>> > >
>>>>>> > > feature
>>>>>> > >
>>>>>> > > >> development on the 1.x line, so I would expect all PRs to
>>>>>> > > >> continue to
>>>>>> > be
>>>>>> > >
>>>>>> > > >> submitted only to main. We would release 1.x as needed with major
>>>>>> > > >> bug
>>>>>> > >
>>>>>> > > >> fixes
>>>>>> > >
>>>>>> > > >> or critical security updates, and these would be cherry-picked
>>>>>> > > >> and/or
>>>>>> > >
>>>>>> > > >> backported as necessary, mostly without the need for PRs, the
>>>>>> > > >> same as
>>>>>> > we
>>>>>> > >
>>>>>> > > >> would do if we were bringing fixes from main (1.20.0-SNAPSHOT)
>>>>>> > > >> back
>>>>>> > to a
>>>>>> > >
>>>>>> > > >> maintenance line like (1.19.x). For precedent, we followed this
>>>>>> > > >> same
>>>>>> > >
>>>>>> > > >> approach going from the 0.x line to 1.0.0 and there wasn't any
>>>>>> > > >> major
>>>>>> > >
>>>>>> > > >> issue.
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >> On Tue, Jan 10, 2023 at 7:07 AM Otto Fowler
>>>>>> > > >> <ot...@gmail.com>
>>>>>> > >
>>>>>> > > >> wrote:
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >> It was also mentioned in another thread that we need to have
>>>>>> > agreement
>>>>>> > >
>>>>>> > > on
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >> our explicit strategy and support for 1.x going forward, did that
>>>>>> > >
>>>>>> > > happen?
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >> From: Otto Fowler <ot...@gmail.com>
>>>>>> > > >> <ot...@gmail.com>
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >> Reply: Otto Fowler <ot...@gmail.com>
>>>>>> > > >> <ottobackwards@gmail.com
>>>>>> > >
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >> Date: January 10, 2023 at 07:02:34
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >> To: dev@nifi.apache.org <de...@nifi.apache.org>
>>>>>> > > >> <de...@nifi.apache.org>
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >> Subject: Re: [discuss] NiFi 1.20 and NiFi 2.0
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >> There needs to be an update to the contributing guide as to how
>>>>>> > > >> to
>>>>>> > >
>>>>>> > > submit
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >> PRs to 1.x or 2.x etc.
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >> From: Joe Witt <jo...@apache.org> <jo...@apache.org>
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >> Reply: dev@nifi.apache.org <de...@nifi.apache.org>
>>>>>> > > >> <dev@nifi.apache.org
>>>>>> > >
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >> Date: January 9, 2023 at 15:53:16
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >> To: dev@nifi.apache.org <de...@nifi.apache.org>
>>>>>> > > >> <de...@nifi.apache.org>
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >> Subject: [discuss] NiFi 1.20 and NiFi 2.0
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >> Team,
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >> As David mentioned in [1] following a successful NiFi 2.0 release
>>>>>> > > >> goal
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >> planning - we are now going to start moving the 'main' line to be
>>>>>> > > >> the
>>>>>> > >
>>>>>> > > NiFi
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >> 2.0 line which will allow for the key work to take place. We will
>>>>>> > > >> also
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >> move niFi 1.x to its appropriate support line.
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >> It is also the case that we have nearly 100 JIRAs on NiFi 1.20
>>>>>> > > >> and we
>>>>>> > >
>>>>>> > > have
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >> work in there including security items so it is time to make a
>>>>>> > release.
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >> The intent then is to initiate 1.20 and immediate after that
>>>>>> > > >> change
>>>>>> > >
>>>>>> > > 'main'
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >> to 2.0.
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >> Going forward then all work on the 1.x line should be focused on
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >> maintaining existing features, dependencies, and helping 1.x
>>>>>> > > >> users
>>>>>> > >
>>>>>> > > migrate
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >> to the 2.x line. Otherwise, new feature work will happen on
>>>>>> > > >> 'main' as
>>>>>> > it
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >> normally does and will come out in the NiFi 2.x release line.
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >> Please flag key outstanding items as we narrow down the release
>>>>>> > >
>>>>>> > > candidate
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >> for NiFi 1.20.
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >> Thanks
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >> Joe
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >> [1]
>>>>>> > > >> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2
>>>>>> > > >> Flists.apache.org%2Fthread%2Fqo4vvdw46235y7vy2crcd6l4m11wl7jz&dat
>>>>>> > > >> a=05%7C01%7Cisha.lamboo%40virtualsciences.nl%7Ccbea974a2c1f479d48
>>>>>> > > >> 9d08daf42521f1%7C21429da9e4ad45f99a6fcd126a64274b%7C0%7C0%7C63809
>>>>>> > > >> 0737572228694%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjo
>>>>>> > > >> iV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=zSqOK9zZPqXxLuxwo0QcqKGEAc7aXjfnnm4i%2BQt2B98%3D&reserved=0
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > > >>
>>>>>> > >
>>>>>> > >
>>>>>> > >
>>>>>> >
Re: [discuss] NiFi 1.20 and NiFi 2.0
Posted by Joe Witt <jo...@gmail.com>.
Team
As you commit to the main line now please start using whatever the latest
relevant feature release is on the 2.x line which right now would be
2.0.0. If we find we need to do a 1.21 and so on we'll pull those commits
down as we would for any other mx release in the past but the 'main' line
becomes 2.0.0 now.
Thanks
Joe
On Mon, Feb 6, 2023 at 10:44 AM Joe Witt <jo...@gmail.com> wrote:
> Team,
>
> I think we are there! Going to kick out RC1 now.
>
> Thanks
>
> On Mon, Jan 30, 2023 at 11:29 AM Joe Witt <jo...@gmail.com> wrote:
>
>> Team,
>>
>> As you can see I've not kicked off the RC yet. Many bug fixes/dependency
>> updates are happening. Ideally we'll wait until nar Maven plugin goes and
>> we're trying to fix some nifi registry/nifi interaction issues as well.
>> Still will get the RC out as soon as we can.
>>
>> Thanks
>>
>> On Mon, Jan 23, 2023 at 11:12 AM Joe Witt <jo...@gmail.com> wrote:
>>
>>> Hello
>>>
>>> Here is a good picture of what the 1.20 RC looks like. I've
>>> tagged several JIRAs today to ensure we get them in. A theme is really
>>> around stabilizing nifi/nifi-registry integration as we're seeing
>>> substantial uptick in usage and thus various community reported findings.
>>> We'll get that quite smooth with these included.
>>>
>>> https://issues.apache.org/jira/projects/NIFI/versions/12352581
>>>
>>> Thanks
>>>
>>> On Mon, Jan 23, 2023 at 8:50 AM Joe Witt <jo...@gmail.com> wrote:
>>>
>>>> Team,
>>>>
>>>> I'm going through the outstanding JIRAs/PRs and flagging which look
>>>> like they should be 'must have' for 1.20 and then will work the RC as soon
>>>> as those land.
>>>>
>>>> Hopefully have the RC up within a day or two but we'll see how these
>>>> land as some have review comments pending action.
>>>>
>>>> Thanks
>>>>
>>>> On Thu, Jan 12, 2023 at 2:53 AM Isha Lamboo <
>>>> isha.lamboo@virtualsciences.nl> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> I would like to contribute to the migration tooling (mostly testing I
>>>>> suppose) when that comes up.
>>>>>
>>>>> My team's largest client has a completely template-based pipeline with
>>>>> external scripts replacing variable values before deploying to target
>>>>> clusters, so we've already started looking at this when the goals for 2.0
>>>>> were discussed and approved. The migration to flowdefinitions and
>>>>> parameters is quite complex and we've hit several blockers when we tried to
>>>>> implement a direct translation.
>>>>>
>>>>> I expect that any time I spend helping to improve the tooling will pay
>>>>> off handsomely for our clients.
>>>>>
>>>>> Regards,
>>>>>
>>>>> Isha
>>>>>
>>>>> -----Oorspronkelijk bericht-----
>>>>> Van: Adam Taft <ad...@adamtaft.com>
>>>>> Verzonden: woensdag 11 januari 2023 23:42
>>>>> Aan: dev@nifi.apache.org
>>>>> Onderwerp: Re: [discuss] NiFi 1.20 and NiFi 2.0
>>>>>
>>>>> This is really insightful and spot on ...
>>>>>
>>>>> Kevin wrote:
>>>>> > Good migration tooling will take a while to develop and test, and
>>>>> the
>>>>> > core contributors to that effort may not have sufficient variety of
>>>>> > flows to evaluate when the migration tools are "done" for the
>>>>> majority
>>>>> > of the community to have success upgrading to 2.x. A milestone
>>>>> release
>>>>> > would
>>>>> allow
>>>>> > us get more feedback on migration over a longer period than the vote
>>>>> window
>>>>> > of an RC candidate.
>>>>>
>>>>> It's exactly this case, that an early 2.0 release might not have had
>>>>> time to fully work its way through existing production deployments, that's
>>>>> concerning. The pace and voting of an "RC" is much too short to get any
>>>>> quality feedback from users in the field.
>>>>>
>>>>> I think it's really smart to consider the "Milestone" release approach
>>>>> here. We release 2.0.0-M1, 2.0.0-M2, ... waiting an adequate amount of time
>>>>> for feedback. We can put these milestones on a calendar, as needed, so that
>>>>> feedback is required some 'x' number of weeks/months after each milestone.
>>>>>
>>>>> And to this end, I'd personally rather see us keep the 'main' branch
>>>>> current with the 1.x line _until_ we're ready and are satisfied with the
>>>>> end goals of the 2.0 release objectives. When the milestone releases have
>>>>> been completed and there's a comfort level with the 2.x line, it's at the
>>>>> point we'd isolate the 1.x line into its own branch and switch main over to
>>>>> the 2.x line.
>>>>>
>>>>> This is an attractive way of:
>>>>> a) continuing business-as-usual with the 1.x line
>>>>> b) making headway on the 2.x release milestones
>>>>> c) giving adequate time for feedback against the 2.0 milestones coming
>>>>> from the field
>>>>>
>>>>> I don't mind the known-unknowns. But it's really the unknown-unknowns
>>>>> that are going to drive a delay in the 2.0 release. I think it's smart to
>>>>> be able to get some of the unknowns ironed out before we finalize the 2.0
>>>>> release ceremony. The milestone approach really helps with that.
>>>>>
>>>>> /Adam
>>>>>
>>>>> On Wed, Jan 11, 2023 at 11:11 AM Kevin Doran <kd...@apache.org>
>>>>> wrote:
>>>>>
>>>>> > Sorry, Joe, I was not clear, and to be honest the two thoughts are
>>>>> > somewhat unrelated in my mind too :)
>>>>> >
>>>>> > I agree that good migration tooling is key. Otherwise, we risk users
>>>>> > staying on 1.x or creating a schism of 1.x and 2.x users.
>>>>> >
>>>>> > Good migration tooling will take a while to develop and test, and
>>>>> the
>>>>> > core contributors to that effort may not have sufficient variety of
>>>>> > flows to evaluate when the migration tools are "done" for the
>>>>> majority
>>>>> > of the community to have success upgrading to 2.x. A milestone
>>>>> release
>>>>> > would allow us get more feedback on migration over a longer period
>>>>> > than the vote window of an RC candidate.
>>>>> >
>>>>> > Perhaps we could continue to release from the 1.x line (including
>>>>> > minor releases with some features) until we are ready to drop the
>>>>> "milestone"
>>>>> > qualifier from 2.0.0, and only then put 1.x into maintenance-only
>>>>> status.
>>>>> > It would be the same proposal to move main to target 2.0.0-M1, but
>>>>> > relax restrictions for what can land on the 1.x branch and be open
>>>>> to
>>>>> > a 1.21, 1.22, etc. if 2.0.0 work takes longer than anticipated. For
>>>>> > example, maybe we would be open to landing new/backported processors
>>>>> > on the 1.x branch, but not core framework features or API changes.
>>>>> >
>>>>> > This might not be necessary, but I think it is fair that saying "no
>>>>> > new features on 1.x" and also "no new features in 2.0.0" puts the
>>>>> > project in a rough place if 2.0.0 takes longer than a few months, so
>>>>> > if we go that route, we need to commit to a quick release of 2.0.0
>>>>> > that most users can move to easily.
>>>>> >
>>>>> > Thanks,
>>>>> > Kevin
>>>>> >
>>>>> > On Jan 11, 2023 at 12:32:46, Joe Witt <jo...@gmail.com> wrote:
>>>>> >
>>>>> > > Kevin,
>>>>> > >
>>>>> > > Yeah we can do whatever we want as far as 'releases' of 2.0 that
>>>>> are
>>>>> > prior
>>>>> > > to us officially considering it 2.0/stable.
>>>>> > >
>>>>> > > That said - the migration tooling will be key to provide as we
>>>>> need
>>>>> > > to
>>>>> > make
>>>>> > > the bridge as solid and stable as possible to help someone move
>>>>> from
>>>>> > > 1.x
>>>>> > to
>>>>> > > 2.x. I dont know how related these two concepts (milestone
>>>>> releases
>>>>> > > and 1.x to 2.x ease really are).
>>>>> > >
>>>>> > > Thanks
>>>>> > >
>>>>> > > On Wed, Jan 11, 2023 at 10:27 AM Kevin Doran <kd...@apache.org>
>>>>> wrote:
>>>>> > >
>>>>> > > [hit the wrong keyboard shortcut, here is the rest of my thoughts]
>>>>> > >
>>>>> > >
>>>>> > > On this point from David:
>>>>> > >
>>>>> > >
>>>>> > > We may need to have a longer release candidate period, or more
>>>>> > incremental
>>>>> > >
>>>>> > > > fix releases
>>>>> > >
>>>>> > > > for the initial 2.0.0 release train, but I do not expect
>>>>> delaying
>>>>> > > > a
>>>>> > 2.0.0
>>>>> > >
>>>>> > > > release for new features, as that is not part of the release
>>>>> goals.
>>>>> > >
>>>>> > > >
>>>>> > >
>>>>> > >
>>>>> > > Would the community benefit from one or more milestone releases of
>>>>> > > 2.0,
>>>>> > to
>>>>> > >
>>>>> > > allow for a wider group to run / live on the proposed 2.0 prior to
>>>>> > >
>>>>> > > releasing it as "stable"? I know we've never done a milestone
>>>>> > > release in
>>>>> > >
>>>>> > > the past, and I'm not sure what ASF guidance is on the topic, but
>>>>> if
>>>>> > > it
>>>>> > >
>>>>> > > could be beneficial we could look into that.
>>>>> > >
>>>>> > >
>>>>> > > Cheers,
>>>>> > >
>>>>> > > Kevin
>>>>> > >
>>>>> > >
>>>>> > > On Jan 11, 2023 at 12:22:43, Kevin Doran <kd...@apache.org>
>>>>> wrote:
>>>>> > >
>>>>> > >
>>>>> > > > I think this is a good, practical discussion.
>>>>> > >
>>>>> > > >
>>>>> > >
>>>>> > > > On the one hand, we can't put off 2.x any longer as we really
>>>>> need
>>>>> > > > to
>>>>> > >
>>>>> > > > updated the minimum required Java to 11. Updating main
>>>>> development
>>>>> > > > to
>>>>> > >
>>>>> > > > target 2.x feels like a good way drive progress on that along
>>>>> with
>>>>> > > > the
>>>>> > >
>>>>> > > > other 2.0 goals.
>>>>> > >
>>>>> > > >
>>>>> > >
>>>>> > > > On the other hand, the concerns are valid: moving all
>>>>> development
>>>>> > > > to
>>>>> > >
>>>>> > > > target 2.x puts the project at risk if we cannot release 2.0.0
>>>>> on
>>>>> > > > a
>>>>> > >
>>>>> > > > reasonable timeline. The restricted scope of 2.0 helps, but this
>>>>> > > > stated
>>>>> > >
>>>>> > > > release goal feels risky to me:
>>>>> > >
>>>>> > > >
>>>>> > >
>>>>> > > > Implement Migration Tools for Upgrading Flows
>>>>> > >
>>>>> > > >
>>>>> > >
>>>>> > > >
>>>>> > >
>>>>> > > > - Implement automated migration where possible to remap
>>>>> > > > properties
>>>>> > and
>>>>> > >
>>>>> > > > features
>>>>> > >
>>>>> > > > - Implement migration tools for manual conversion of XML
>>>>> > Templates
>>>>> > >
>>>>> > > > to JSON Flow Definitions
>>>>> > >
>>>>> > > > - Create documentation for manual steps necessary where
>>>>> > >
>>>>> > > > programmatic migration cannot be implemented
>>>>> > >
>>>>> > > > - NiFi 2.0 should be capable of starting with ghosted
>>>>> > > > components
>>>>> > >
>>>>> > > > for removed Processors or Controller Services
>>>>> > >
>>>>> > > >
>>>>> > >
>>>>> > > >
>>>>> > >
>>>>> > > > Removing deprecated components should be fairly straightforward
>>>>> > > > and
>>>>> > >
>>>>> > > quick,
>>>>> > >
>>>>> > > > but automating and documenting migration is a large effort.
>>>>> > >
>>>>> > > >
>>>>> > >
>>>>> > > > On this po
>>>>> > >
>>>>> > > >
>>>>> > >
>>>>> > > >
>>>>> > >
>>>>> > > > On Jan 10, 2023 at 09:32:31, Bryan Bende <bb...@gmail.com>
>>>>> wrote:
>>>>> > >
>>>>> > > >
>>>>> > >
>>>>> > > >> The plan as I understand it is not to diverge and create
>>>>> separate
>>>>> > >
>>>>> > > feature
>>>>> > >
>>>>> > > >> development on the 1.x line, so I would expect all PRs to
>>>>> > > >> continue to
>>>>> > be
>>>>> > >
>>>>> > > >> submitted only to main. We would release 1.x as needed with
>>>>> major
>>>>> > > >> bug
>>>>> > >
>>>>> > > >> fixes
>>>>> > >
>>>>> > > >> or critical security updates, and these would be cherry-picked
>>>>> > > >> and/or
>>>>> > >
>>>>> > > >> backported as necessary, mostly without the need for PRs, the
>>>>> > > >> same as
>>>>> > we
>>>>> > >
>>>>> > > >> would do if we were bringing fixes from main (1.20.0-SNAPSHOT)
>>>>> > > >> back
>>>>> > to a
>>>>> > >
>>>>> > > >> maintenance line like (1.19.x). For precedent, we followed this
>>>>> > > >> same
>>>>> > >
>>>>> > > >> approach going from the 0.x line to 1.0.0 and there wasn't any
>>>>> > > >> major
>>>>> > >
>>>>> > > >> issue.
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >> On Tue, Jan 10, 2023 at 7:07 AM Otto Fowler
>>>>> > > >> <ot...@gmail.com>
>>>>> > >
>>>>> > > >> wrote:
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >> It was also mentioned in another thread that we need to have
>>>>> > agreement
>>>>> > >
>>>>> > > on
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >> our explicit strategy and support for 1.x going forward, did
>>>>> that
>>>>> > >
>>>>> > > happen?
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >> From: Otto Fowler <ot...@gmail.com>
>>>>> > > >> <ot...@gmail.com>
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >> Reply: Otto Fowler <ot...@gmail.com>
>>>>> > > >> <ottobackwards@gmail.com
>>>>> > >
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >> Date: January 10, 2023 at 07:02:34
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >> To: dev@nifi.apache.org <de...@nifi.apache.org>
>>>>> > > >> <de...@nifi.apache.org>
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >> Subject: Re: [discuss] NiFi 1.20 and NiFi 2.0
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >> There needs to be an update to the contributing guide as to how
>>>>> > > >> to
>>>>> > >
>>>>> > > submit
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >> PRs to 1.x or 2.x etc.
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >> From: Joe Witt <jo...@apache.org> <jo...@apache.org>
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >> Reply: dev@nifi.apache.org <de...@nifi.apache.org>
>>>>> > > >> <dev@nifi.apache.org
>>>>> > >
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >> Date: January 9, 2023 at 15:53:16
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >> To: dev@nifi.apache.org <de...@nifi.apache.org>
>>>>> > > >> <de...@nifi.apache.org>
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >> Subject: [discuss] NiFi 1.20 and NiFi 2.0
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >> Team,
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >> As David mentioned in [1] following a successful NiFi 2.0
>>>>> release
>>>>> > > >> goal
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >> planning - we are now going to start moving the 'main' line to
>>>>> be
>>>>> > > >> the
>>>>> > >
>>>>> > > NiFi
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >> 2.0 line which will allow for the key work to take place. We
>>>>> will
>>>>> > > >> also
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >> move niFi 1.x to its appropriate support line.
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >> It is also the case that we have nearly 100 JIRAs on NiFi 1.20
>>>>> > > >> and we
>>>>> > >
>>>>> > > have
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >> work in there including security items so it is time to make a
>>>>> > release.
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >> The intent then is to initiate 1.20 and immediate after that
>>>>> > > >> change
>>>>> > >
>>>>> > > 'main'
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >> to 2.0.
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >> Going forward then all work on the 1.x line should be focused on
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >> maintaining existing features, dependencies, and helping 1.x
>>>>> > > >> users
>>>>> > >
>>>>> > > migrate
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >> to the 2.x line. Otherwise, new feature work will happen on
>>>>> > > >> 'main' as
>>>>> > it
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >> normally does and will come out in the NiFi 2.x release line.
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >> Please flag key outstanding items as we narrow down the release
>>>>> > >
>>>>> > > candidate
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >> for NiFi 1.20.
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >> Thanks
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >> Joe
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >> [1]
>>>>> > > >>
>>>>> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2
>>>>> > > >> Flists.apache.org
>>>>> %2Fthread%2Fqo4vvdw46235y7vy2crcd6l4m11wl7jz&dat
>>>>> > > >> a=05%7C01%7Cisha.lamboo%40virtualsciences.nl
>>>>> %7Ccbea974a2c1f479d48
>>>>> > > >>
>>>>> 9d08daf42521f1%7C21429da9e4ad45f99a6fcd126a64274b%7C0%7C0%7C63809
>>>>> > > >>
>>>>> 0737572228694%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjo
>>>>> > > >>
>>>>> iV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=zSqOK9zZPqXxLuxwo0QcqKGEAc7aXjfnnm4i%2BQt2B98%3D&reserved=0
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > > >>
>>>>> > >
>>>>> > >
>>>>> > >
>>>>> >
>>>>>
>>>>
Re: [discuss] NiFi 1.20 and NiFi 2.0
Posted by Joe Witt <jo...@gmail.com>.
Team,
I think we are there! Going to kick out RC1 now.
Thanks
On Mon, Jan 30, 2023 at 11:29 AM Joe Witt <jo...@gmail.com> wrote:
> Team,
>
> As you can see I've not kicked off the RC yet. Many bug fixes/dependency
> updates are happening. Ideally we'll wait until nar Maven plugin goes and
> we're trying to fix some nifi registry/nifi interaction issues as well.
> Still will get the RC out as soon as we can.
>
> Thanks
>
> On Mon, Jan 23, 2023 at 11:12 AM Joe Witt <jo...@gmail.com> wrote:
>
>> Hello
>>
>> Here is a good picture of what the 1.20 RC looks like. I've
>> tagged several JIRAs today to ensure we get them in. A theme is really
>> around stabilizing nifi/nifi-registry integration as we're seeing
>> substantial uptick in usage and thus various community reported findings.
>> We'll get that quite smooth with these included.
>>
>> https://issues.apache.org/jira/projects/NIFI/versions/12352581
>>
>> Thanks
>>
>> On Mon, Jan 23, 2023 at 8:50 AM Joe Witt <jo...@gmail.com> wrote:
>>
>>> Team,
>>>
>>> I'm going through the outstanding JIRAs/PRs and flagging which look like
>>> they should be 'must have' for 1.20 and then will work the RC as soon as
>>> those land.
>>>
>>> Hopefully have the RC up within a day or two but we'll see how these
>>> land as some have review comments pending action.
>>>
>>> Thanks
>>>
>>> On Thu, Jan 12, 2023 at 2:53 AM Isha Lamboo <
>>> isha.lamboo@virtualsciences.nl> wrote:
>>>
>>>> Hi all,
>>>>
>>>> I would like to contribute to the migration tooling (mostly testing I
>>>> suppose) when that comes up.
>>>>
>>>> My team's largest client has a completely template-based pipeline with
>>>> external scripts replacing variable values before deploying to target
>>>> clusters, so we've already started looking at this when the goals for 2.0
>>>> were discussed and approved. The migration to flowdefinitions and
>>>> parameters is quite complex and we've hit several blockers when we tried to
>>>> implement a direct translation.
>>>>
>>>> I expect that any time I spend helping to improve the tooling will pay
>>>> off handsomely for our clients.
>>>>
>>>> Regards,
>>>>
>>>> Isha
>>>>
>>>> -----Oorspronkelijk bericht-----
>>>> Van: Adam Taft <ad...@adamtaft.com>
>>>> Verzonden: woensdag 11 januari 2023 23:42
>>>> Aan: dev@nifi.apache.org
>>>> Onderwerp: Re: [discuss] NiFi 1.20 and NiFi 2.0
>>>>
>>>> This is really insightful and spot on ...
>>>>
>>>> Kevin wrote:
>>>> > Good migration tooling will take a while to develop and test, and the
>>>> > core contributors to that effort may not have sufficient variety of
>>>> > flows to evaluate when the migration tools are "done" for the
>>>> majority
>>>> > of the community to have success upgrading to 2.x. A milestone
>>>> release
>>>> > would
>>>> allow
>>>> > us get more feedback on migration over a longer period than the vote
>>>> window
>>>> > of an RC candidate.
>>>>
>>>> It's exactly this case, that an early 2.0 release might not have had
>>>> time to fully work its way through existing production deployments, that's
>>>> concerning. The pace and voting of an "RC" is much too short to get any
>>>> quality feedback from users in the field.
>>>>
>>>> I think it's really smart to consider the "Milestone" release approach
>>>> here. We release 2.0.0-M1, 2.0.0-M2, ... waiting an adequate amount of time
>>>> for feedback. We can put these milestones on a calendar, as needed, so that
>>>> feedback is required some 'x' number of weeks/months after each milestone.
>>>>
>>>> And to this end, I'd personally rather see us keep the 'main' branch
>>>> current with the 1.x line _until_ we're ready and are satisfied with the
>>>> end goals of the 2.0 release objectives. When the milestone releases have
>>>> been completed and there's a comfort level with the 2.x line, it's at the
>>>> point we'd isolate the 1.x line into its own branch and switch main over to
>>>> the 2.x line.
>>>>
>>>> This is an attractive way of:
>>>> a) continuing business-as-usual with the 1.x line
>>>> b) making headway on the 2.x release milestones
>>>> c) giving adequate time for feedback against the 2.0 milestones coming
>>>> from the field
>>>>
>>>> I don't mind the known-unknowns. But it's really the unknown-unknowns
>>>> that are going to drive a delay in the 2.0 release. I think it's smart to
>>>> be able to get some of the unknowns ironed out before we finalize the 2.0
>>>> release ceremony. The milestone approach really helps with that.
>>>>
>>>> /Adam
>>>>
>>>> On Wed, Jan 11, 2023 at 11:11 AM Kevin Doran <kd...@apache.org> wrote:
>>>>
>>>> > Sorry, Joe, I was not clear, and to be honest the two thoughts are
>>>> > somewhat unrelated in my mind too :)
>>>> >
>>>> > I agree that good migration tooling is key. Otherwise, we risk users
>>>> > staying on 1.x or creating a schism of 1.x and 2.x users.
>>>> >
>>>> > Good migration tooling will take a while to develop and test, and the
>>>> > core contributors to that effort may not have sufficient variety of
>>>> > flows to evaluate when the migration tools are "done" for the
>>>> majority
>>>> > of the community to have success upgrading to 2.x. A milestone
>>>> release
>>>> > would allow us get more feedback on migration over a longer period
>>>> > than the vote window of an RC candidate.
>>>> >
>>>> > Perhaps we could continue to release from the 1.x line (including
>>>> > minor releases with some features) until we are ready to drop the
>>>> "milestone"
>>>> > qualifier from 2.0.0, and only then put 1.x into maintenance-only
>>>> status.
>>>> > It would be the same proposal to move main to target 2.0.0-M1, but
>>>> > relax restrictions for what can land on the 1.x branch and be open to
>>>> > a 1.21, 1.22, etc. if 2.0.0 work takes longer than anticipated. For
>>>> > example, maybe we would be open to landing new/backported processors
>>>> > on the 1.x branch, but not core framework features or API changes.
>>>> >
>>>> > This might not be necessary, but I think it is fair that saying "no
>>>> > new features on 1.x" and also "no new features in 2.0.0" puts the
>>>> > project in a rough place if 2.0.0 takes longer than a few months, so
>>>> > if we go that route, we need to commit to a quick release of 2.0.0
>>>> > that most users can move to easily.
>>>> >
>>>> > Thanks,
>>>> > Kevin
>>>> >
>>>> > On Jan 11, 2023 at 12:32:46, Joe Witt <jo...@gmail.com> wrote:
>>>> >
>>>> > > Kevin,
>>>> > >
>>>> > > Yeah we can do whatever we want as far as 'releases' of 2.0 that are
>>>> > prior
>>>> > > to us officially considering it 2.0/stable.
>>>> > >
>>>> > > That said - the migration tooling will be key to provide as we need
>>>> > > to
>>>> > make
>>>> > > the bridge as solid and stable as possible to help someone move
>>>> from
>>>> > > 1.x
>>>> > to
>>>> > > 2.x. I dont know how related these two concepts (milestone
>>>> releases
>>>> > > and 1.x to 2.x ease really are).
>>>> > >
>>>> > > Thanks
>>>> > >
>>>> > > On Wed, Jan 11, 2023 at 10:27 AM Kevin Doran <kd...@apache.org>
>>>> wrote:
>>>> > >
>>>> > > [hit the wrong keyboard shortcut, here is the rest of my thoughts]
>>>> > >
>>>> > >
>>>> > > On this point from David:
>>>> > >
>>>> > >
>>>> > > We may need to have a longer release candidate period, or more
>>>> > incremental
>>>> > >
>>>> > > > fix releases
>>>> > >
>>>> > > > for the initial 2.0.0 release train, but I do not expect delaying
>>>> > > > a
>>>> > 2.0.0
>>>> > >
>>>> > > > release for new features, as that is not part of the release
>>>> goals.
>>>> > >
>>>> > > >
>>>> > >
>>>> > >
>>>> > > Would the community benefit from one or more milestone releases of
>>>> > > 2.0,
>>>> > to
>>>> > >
>>>> > > allow for a wider group to run / live on the proposed 2.0 prior to
>>>> > >
>>>> > > releasing it as "stable"? I know we've never done a milestone
>>>> > > release in
>>>> > >
>>>> > > the past, and I'm not sure what ASF guidance is on the topic, but
>>>> if
>>>> > > it
>>>> > >
>>>> > > could be beneficial we could look into that.
>>>> > >
>>>> > >
>>>> > > Cheers,
>>>> > >
>>>> > > Kevin
>>>> > >
>>>> > >
>>>> > > On Jan 11, 2023 at 12:22:43, Kevin Doran <kd...@apache.org> wrote:
>>>> > >
>>>> > >
>>>> > > > I think this is a good, practical discussion.
>>>> > >
>>>> > > >
>>>> > >
>>>> > > > On the one hand, we can't put off 2.x any longer as we really
>>>> need
>>>> > > > to
>>>> > >
>>>> > > > updated the minimum required Java to 11. Updating main
>>>> development
>>>> > > > to
>>>> > >
>>>> > > > target 2.x feels like a good way drive progress on that along
>>>> with
>>>> > > > the
>>>> > >
>>>> > > > other 2.0 goals.
>>>> > >
>>>> > > >
>>>> > >
>>>> > > > On the other hand, the concerns are valid: moving all development
>>>> > > > to
>>>> > >
>>>> > > > target 2.x puts the project at risk if we cannot release 2.0.0 on
>>>> > > > a
>>>> > >
>>>> > > > reasonable timeline. The restricted scope of 2.0 helps, but this
>>>> > > > stated
>>>> > >
>>>> > > > release goal feels risky to me:
>>>> > >
>>>> > > >
>>>> > >
>>>> > > > Implement Migration Tools for Upgrading Flows
>>>> > >
>>>> > > >
>>>> > >
>>>> > > >
>>>> > >
>>>> > > > - Implement automated migration where possible to remap
>>>> > > > properties
>>>> > and
>>>> > >
>>>> > > > features
>>>> > >
>>>> > > > - Implement migration tools for manual conversion of XML
>>>> > Templates
>>>> > >
>>>> > > > to JSON Flow Definitions
>>>> > >
>>>> > > > - Create documentation for manual steps necessary where
>>>> > >
>>>> > > > programmatic migration cannot be implemented
>>>> > >
>>>> > > > - NiFi 2.0 should be capable of starting with ghosted
>>>> > > > components
>>>> > >
>>>> > > > for removed Processors or Controller Services
>>>> > >
>>>> > > >
>>>> > >
>>>> > > >
>>>> > >
>>>> > > > Removing deprecated components should be fairly straightforward
>>>> > > > and
>>>> > >
>>>> > > quick,
>>>> > >
>>>> > > > but automating and documenting migration is a large effort.
>>>> > >
>>>> > > >
>>>> > >
>>>> > > > On this po
>>>> > >
>>>> > > >
>>>> > >
>>>> > > >
>>>> > >
>>>> > > > On Jan 10, 2023 at 09:32:31, Bryan Bende <bb...@gmail.com>
>>>> wrote:
>>>> > >
>>>> > > >
>>>> > >
>>>> > > >> The plan as I understand it is not to diverge and create separate
>>>> > >
>>>> > > feature
>>>> > >
>>>> > > >> development on the 1.x line, so I would expect all PRs to
>>>> > > >> continue to
>>>> > be
>>>> > >
>>>> > > >> submitted only to main. We would release 1.x as needed with
>>>> major
>>>> > > >> bug
>>>> > >
>>>> > > >> fixes
>>>> > >
>>>> > > >> or critical security updates, and these would be cherry-picked
>>>> > > >> and/or
>>>> > >
>>>> > > >> backported as necessary, mostly without the need for PRs, the
>>>> > > >> same as
>>>> > we
>>>> > >
>>>> > > >> would do if we were bringing fixes from main (1.20.0-SNAPSHOT)
>>>> > > >> back
>>>> > to a
>>>> > >
>>>> > > >> maintenance line like (1.19.x). For precedent, we followed this
>>>> > > >> same
>>>> > >
>>>> > > >> approach going from the 0.x line to 1.0.0 and there wasn't any
>>>> > > >> major
>>>> > >
>>>> > > >> issue.
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >> On Tue, Jan 10, 2023 at 7:07 AM Otto Fowler
>>>> > > >> <ot...@gmail.com>
>>>> > >
>>>> > > >> wrote:
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >> It was also mentioned in another thread that we need to have
>>>> > agreement
>>>> > >
>>>> > > on
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >> our explicit strategy and support for 1.x going forward, did that
>>>> > >
>>>> > > happen?
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >> From: Otto Fowler <ot...@gmail.com>
>>>> > > >> <ot...@gmail.com>
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >> Reply: Otto Fowler <ot...@gmail.com>
>>>> > > >> <ottobackwards@gmail.com
>>>> > >
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >> Date: January 10, 2023 at 07:02:34
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >> To: dev@nifi.apache.org <de...@nifi.apache.org>
>>>> > > >> <de...@nifi.apache.org>
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >> Subject: Re: [discuss] NiFi 1.20 and NiFi 2.0
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >> There needs to be an update to the contributing guide as to how
>>>> > > >> to
>>>> > >
>>>> > > submit
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >> PRs to 1.x or 2.x etc.
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >> From: Joe Witt <jo...@apache.org> <jo...@apache.org>
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >> Reply: dev@nifi.apache.org <de...@nifi.apache.org>
>>>> > > >> <dev@nifi.apache.org
>>>> > >
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >> Date: January 9, 2023 at 15:53:16
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >> To: dev@nifi.apache.org <de...@nifi.apache.org>
>>>> > > >> <de...@nifi.apache.org>
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >> Subject: [discuss] NiFi 1.20 and NiFi 2.0
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >> Team,
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >> As David mentioned in [1] following a successful NiFi 2.0
>>>> release
>>>> > > >> goal
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >> planning - we are now going to start moving the 'main' line to
>>>> be
>>>> > > >> the
>>>> > >
>>>> > > NiFi
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >> 2.0 line which will allow for the key work to take place. We
>>>> will
>>>> > > >> also
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >> move niFi 1.x to its appropriate support line.
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >> It is also the case that we have nearly 100 JIRAs on NiFi 1.20
>>>> > > >> and we
>>>> > >
>>>> > > have
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >> work in there including security items so it is time to make a
>>>> > release.
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >> The intent then is to initiate 1.20 and immediate after that
>>>> > > >> change
>>>> > >
>>>> > > 'main'
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >> to 2.0.
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >> Going forward then all work on the 1.x line should be focused on
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >> maintaining existing features, dependencies, and helping 1.x
>>>> > > >> users
>>>> > >
>>>> > > migrate
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >> to the 2.x line. Otherwise, new feature work will happen on
>>>> > > >> 'main' as
>>>> > it
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >> normally does and will come out in the NiFi 2.x release line.
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >> Please flag key outstanding items as we narrow down the release
>>>> > >
>>>> > > candidate
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >> for NiFi 1.20.
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >> Thanks
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >> Joe
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >> [1]
>>>> > > >>
>>>> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2
>>>> > > >> Flists.apache.org
>>>> %2Fthread%2Fqo4vvdw46235y7vy2crcd6l4m11wl7jz&dat
>>>> > > >> a=05%7C01%7Cisha.lamboo%40virtualsciences.nl
>>>> %7Ccbea974a2c1f479d48
>>>> > > >> 9d08daf42521f1%7C21429da9e4ad45f99a6fcd126a64274b%7C0%7C0%7C63809
>>>> > > >> 0737572228694%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjo
>>>> > > >>
>>>> iV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=zSqOK9zZPqXxLuxwo0QcqKGEAc7aXjfnnm4i%2BQt2B98%3D&reserved=0
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >>
>>>> > >
>>>> > > >>
>>>> > >
>>>> > >
>>>> > >
>>>> >
>>>>
>>>
Re: [discuss] NiFi 1.20 and NiFi 2.0
Posted by Joe Witt <jo...@gmail.com>.
Team,
As you can see I've not kicked off the RC yet. Many bug fixes/dependency
updates are happening. Ideally we'll wait until nar Maven plugin goes and
we're trying to fix some nifi registry/nifi interaction issues as well.
Still will get the RC out as soon as we can.
Thanks
On Mon, Jan 23, 2023 at 11:12 AM Joe Witt <jo...@gmail.com> wrote:
> Hello
>
> Here is a good picture of what the 1.20 RC looks like. I've
> tagged several JIRAs today to ensure we get them in. A theme is really
> around stabilizing nifi/nifi-registry integration as we're seeing
> substantial uptick in usage and thus various community reported findings.
> We'll get that quite smooth with these included.
>
> https://issues.apache.org/jira/projects/NIFI/versions/12352581
>
> Thanks
>
> On Mon, Jan 23, 2023 at 8:50 AM Joe Witt <jo...@gmail.com> wrote:
>
>> Team,
>>
>> I'm going through the outstanding JIRAs/PRs and flagging which look like
>> they should be 'must have' for 1.20 and then will work the RC as soon as
>> those land.
>>
>> Hopefully have the RC up within a day or two but we'll see how these land
>> as some have review comments pending action.
>>
>> Thanks
>>
>> On Thu, Jan 12, 2023 at 2:53 AM Isha Lamboo <
>> isha.lamboo@virtualsciences.nl> wrote:
>>
>>> Hi all,
>>>
>>> I would like to contribute to the migration tooling (mostly testing I
>>> suppose) when that comes up.
>>>
>>> My team's largest client has a completely template-based pipeline with
>>> external scripts replacing variable values before deploying to target
>>> clusters, so we've already started looking at this when the goals for 2.0
>>> were discussed and approved. The migration to flowdefinitions and
>>> parameters is quite complex and we've hit several blockers when we tried to
>>> implement a direct translation.
>>>
>>> I expect that any time I spend helping to improve the tooling will pay
>>> off handsomely for our clients.
>>>
>>> Regards,
>>>
>>> Isha
>>>
>>> -----Oorspronkelijk bericht-----
>>> Van: Adam Taft <ad...@adamtaft.com>
>>> Verzonden: woensdag 11 januari 2023 23:42
>>> Aan: dev@nifi.apache.org
>>> Onderwerp: Re: [discuss] NiFi 1.20 and NiFi 2.0
>>>
>>> This is really insightful and spot on ...
>>>
>>> Kevin wrote:
>>> > Good migration tooling will take a while to develop and test, and the
>>> > core contributors to that effort may not have sufficient variety of
>>> > flows to evaluate when the migration tools are "done" for the majority
>>> > of the community to have success upgrading to 2.x. A milestone release
>>> > would
>>> allow
>>> > us get more feedback on migration over a longer period than the vote
>>> window
>>> > of an RC candidate.
>>>
>>> It's exactly this case, that an early 2.0 release might not have had
>>> time to fully work its way through existing production deployments, that's
>>> concerning. The pace and voting of an "RC" is much too short to get any
>>> quality feedback from users in the field.
>>>
>>> I think it's really smart to consider the "Milestone" release approach
>>> here. We release 2.0.0-M1, 2.0.0-M2, ... waiting an adequate amount of time
>>> for feedback. We can put these milestones on a calendar, as needed, so that
>>> feedback is required some 'x' number of weeks/months after each milestone.
>>>
>>> And to this end, I'd personally rather see us keep the 'main' branch
>>> current with the 1.x line _until_ we're ready and are satisfied with the
>>> end goals of the 2.0 release objectives. When the milestone releases have
>>> been completed and there's a comfort level with the 2.x line, it's at the
>>> point we'd isolate the 1.x line into its own branch and switch main over to
>>> the 2.x line.
>>>
>>> This is an attractive way of:
>>> a) continuing business-as-usual with the 1.x line
>>> b) making headway on the 2.x release milestones
>>> c) giving adequate time for feedback against the 2.0 milestones coming
>>> from the field
>>>
>>> I don't mind the known-unknowns. But it's really the unknown-unknowns
>>> that are going to drive a delay in the 2.0 release. I think it's smart to
>>> be able to get some of the unknowns ironed out before we finalize the 2.0
>>> release ceremony. The milestone approach really helps with that.
>>>
>>> /Adam
>>>
>>> On Wed, Jan 11, 2023 at 11:11 AM Kevin Doran <kd...@apache.org> wrote:
>>>
>>> > Sorry, Joe, I was not clear, and to be honest the two thoughts are
>>> > somewhat unrelated in my mind too :)
>>> >
>>> > I agree that good migration tooling is key. Otherwise, we risk users
>>> > staying on 1.x or creating a schism of 1.x and 2.x users.
>>> >
>>> > Good migration tooling will take a while to develop and test, and the
>>> > core contributors to that effort may not have sufficient variety of
>>> > flows to evaluate when the migration tools are "done" for the majority
>>> > of the community to have success upgrading to 2.x. A milestone release
>>> > would allow us get more feedback on migration over a longer period
>>> > than the vote window of an RC candidate.
>>> >
>>> > Perhaps we could continue to release from the 1.x line (including
>>> > minor releases with some features) until we are ready to drop the
>>> "milestone"
>>> > qualifier from 2.0.0, and only then put 1.x into maintenance-only
>>> status.
>>> > It would be the same proposal to move main to target 2.0.0-M1, but
>>> > relax restrictions for what can land on the 1.x branch and be open to
>>> > a 1.21, 1.22, etc. if 2.0.0 work takes longer than anticipated. For
>>> > example, maybe we would be open to landing new/backported processors
>>> > on the 1.x branch, but not core framework features or API changes.
>>> >
>>> > This might not be necessary, but I think it is fair that saying "no
>>> > new features on 1.x" and also "no new features in 2.0.0" puts the
>>> > project in a rough place if 2.0.0 takes longer than a few months, so
>>> > if we go that route, we need to commit to a quick release of 2.0.0
>>> > that most users can move to easily.
>>> >
>>> > Thanks,
>>> > Kevin
>>> >
>>> > On Jan 11, 2023 at 12:32:46, Joe Witt <jo...@gmail.com> wrote:
>>> >
>>> > > Kevin,
>>> > >
>>> > > Yeah we can do whatever we want as far as 'releases' of 2.0 that are
>>> > prior
>>> > > to us officially considering it 2.0/stable.
>>> > >
>>> > > That said - the migration tooling will be key to provide as we need
>>> > > to
>>> > make
>>> > > the bridge as solid and stable as possible to help someone move from
>>> > > 1.x
>>> > to
>>> > > 2.x. I dont know how related these two concepts (milestone releases
>>> > > and 1.x to 2.x ease really are).
>>> > >
>>> > > Thanks
>>> > >
>>> > > On Wed, Jan 11, 2023 at 10:27 AM Kevin Doran <kd...@apache.org>
>>> wrote:
>>> > >
>>> > > [hit the wrong keyboard shortcut, here is the rest of my thoughts]
>>> > >
>>> > >
>>> > > On this point from David:
>>> > >
>>> > >
>>> > > We may need to have a longer release candidate period, or more
>>> > incremental
>>> > >
>>> > > > fix releases
>>> > >
>>> > > > for the initial 2.0.0 release train, but I do not expect delaying
>>> > > > a
>>> > 2.0.0
>>> > >
>>> > > > release for new features, as that is not part of the release goals.
>>> > >
>>> > > >
>>> > >
>>> > >
>>> > > Would the community benefit from one or more milestone releases of
>>> > > 2.0,
>>> > to
>>> > >
>>> > > allow for a wider group to run / live on the proposed 2.0 prior to
>>> > >
>>> > > releasing it as "stable"? I know we've never done a milestone
>>> > > release in
>>> > >
>>> > > the past, and I'm not sure what ASF guidance is on the topic, but if
>>> > > it
>>> > >
>>> > > could be beneficial we could look into that.
>>> > >
>>> > >
>>> > > Cheers,
>>> > >
>>> > > Kevin
>>> > >
>>> > >
>>> > > On Jan 11, 2023 at 12:22:43, Kevin Doran <kd...@apache.org> wrote:
>>> > >
>>> > >
>>> > > > I think this is a good, practical discussion.
>>> > >
>>> > > >
>>> > >
>>> > > > On the one hand, we can't put off 2.x any longer as we really need
>>> > > > to
>>> > >
>>> > > > updated the minimum required Java to 11. Updating main development
>>> > > > to
>>> > >
>>> > > > target 2.x feels like a good way drive progress on that along with
>>> > > > the
>>> > >
>>> > > > other 2.0 goals.
>>> > >
>>> > > >
>>> > >
>>> > > > On the other hand, the concerns are valid: moving all development
>>> > > > to
>>> > >
>>> > > > target 2.x puts the project at risk if we cannot release 2.0.0 on
>>> > > > a
>>> > >
>>> > > > reasonable timeline. The restricted scope of 2.0 helps, but this
>>> > > > stated
>>> > >
>>> > > > release goal feels risky to me:
>>> > >
>>> > > >
>>> > >
>>> > > > Implement Migration Tools for Upgrading Flows
>>> > >
>>> > > >
>>> > >
>>> > > >
>>> > >
>>> > > > - Implement automated migration where possible to remap
>>> > > > properties
>>> > and
>>> > >
>>> > > > features
>>> > >
>>> > > > - Implement migration tools for manual conversion of XML
>>> > Templates
>>> > >
>>> > > > to JSON Flow Definitions
>>> > >
>>> > > > - Create documentation for manual steps necessary where
>>> > >
>>> > > > programmatic migration cannot be implemented
>>> > >
>>> > > > - NiFi 2.0 should be capable of starting with ghosted
>>> > > > components
>>> > >
>>> > > > for removed Processors or Controller Services
>>> > >
>>> > > >
>>> > >
>>> > > >
>>> > >
>>> > > > Removing deprecated components should be fairly straightforward
>>> > > > and
>>> > >
>>> > > quick,
>>> > >
>>> > > > but automating and documenting migration is a large effort.
>>> > >
>>> > > >
>>> > >
>>> > > > On this po
>>> > >
>>> > > >
>>> > >
>>> > > >
>>> > >
>>> > > > On Jan 10, 2023 at 09:32:31, Bryan Bende <bb...@gmail.com> wrote:
>>> > >
>>> > > >
>>> > >
>>> > > >> The plan as I understand it is not to diverge and create separate
>>> > >
>>> > > feature
>>> > >
>>> > > >> development on the 1.x line, so I would expect all PRs to
>>> > > >> continue to
>>> > be
>>> > >
>>> > > >> submitted only to main. We would release 1.x as needed with major
>>> > > >> bug
>>> > >
>>> > > >> fixes
>>> > >
>>> > > >> or critical security updates, and these would be cherry-picked
>>> > > >> and/or
>>> > >
>>> > > >> backported as necessary, mostly without the need for PRs, the
>>> > > >> same as
>>> > we
>>> > >
>>> > > >> would do if we were bringing fixes from main (1.20.0-SNAPSHOT)
>>> > > >> back
>>> > to a
>>> > >
>>> > > >> maintenance line like (1.19.x). For precedent, we followed this
>>> > > >> same
>>> > >
>>> > > >> approach going from the 0.x line to 1.0.0 and there wasn't any
>>> > > >> major
>>> > >
>>> > > >> issue.
>>> > >
>>> > > >>
>>> > >
>>> > > >> On Tue, Jan 10, 2023 at 7:07 AM Otto Fowler
>>> > > >> <ot...@gmail.com>
>>> > >
>>> > > >> wrote:
>>> > >
>>> > > >>
>>> > >
>>> > > >> It was also mentioned in another thread that we need to have
>>> > agreement
>>> > >
>>> > > on
>>> > >
>>> > > >>
>>> > >
>>> > > >> our explicit strategy and support for 1.x going forward, did that
>>> > >
>>> > > happen?
>>> > >
>>> > > >>
>>> > >
>>> > > >>
>>> > >
>>> > > >> From: Otto Fowler <ot...@gmail.com>
>>> > > >> <ot...@gmail.com>
>>> > >
>>> > > >>
>>> > >
>>> > > >> Reply: Otto Fowler <ot...@gmail.com>
>>> > > >> <ottobackwards@gmail.com
>>> > >
>>> > >
>>> > > >>
>>> > >
>>> > > >> Date: January 10, 2023 at 07:02:34
>>> > >
>>> > > >>
>>> > >
>>> > > >> To: dev@nifi.apache.org <de...@nifi.apache.org>
>>> > > >> <de...@nifi.apache.org>
>>> > >
>>> > > >>
>>> > >
>>> > > >> Subject: Re: [discuss] NiFi 1.20 and NiFi 2.0
>>> > >
>>> > > >>
>>> > >
>>> > > >>
>>> > >
>>> > > >> There needs to be an update to the contributing guide as to how
>>> > > >> to
>>> > >
>>> > > submit
>>> > >
>>> > > >>
>>> > >
>>> > > >> PRs to 1.x or 2.x etc.
>>> > >
>>> > > >>
>>> > >
>>> > > >>
>>> > >
>>> > > >> From: Joe Witt <jo...@apache.org> <jo...@apache.org>
>>> > >
>>> > > >>
>>> > >
>>> > > >> Reply: dev@nifi.apache.org <de...@nifi.apache.org>
>>> > > >> <dev@nifi.apache.org
>>> > >
>>> > >
>>> > > >>
>>> > >
>>> > > >> Date: January 9, 2023 at 15:53:16
>>> > >
>>> > > >>
>>> > >
>>> > > >> To: dev@nifi.apache.org <de...@nifi.apache.org>
>>> > > >> <de...@nifi.apache.org>
>>> > >
>>> > > >>
>>> > >
>>> > > >> Subject: [discuss] NiFi 1.20 and NiFi 2.0
>>> > >
>>> > > >>
>>> > >
>>> > > >>
>>> > >
>>> > > >> Team,
>>> > >
>>> > > >>
>>> > >
>>> > > >>
>>> > >
>>> > > >> As David mentioned in [1] following a successful NiFi 2.0 release
>>> > > >> goal
>>> > >
>>> > > >>
>>> > >
>>> > > >> planning - we are now going to start moving the 'main' line to be
>>> > > >> the
>>> > >
>>> > > NiFi
>>> > >
>>> > > >>
>>> > >
>>> > > >> 2.0 line which will allow for the key work to take place. We will
>>> > > >> also
>>> > >
>>> > > >>
>>> > >
>>> > > >> move niFi 1.x to its appropriate support line.
>>> > >
>>> > > >>
>>> > >
>>> > > >>
>>> > >
>>> > > >> It is also the case that we have nearly 100 JIRAs on NiFi 1.20
>>> > > >> and we
>>> > >
>>> > > have
>>> > >
>>> > > >>
>>> > >
>>> > > >> work in there including security items so it is time to make a
>>> > release.
>>> > >
>>> > > >>
>>> > >
>>> > > >> The intent then is to initiate 1.20 and immediate after that
>>> > > >> change
>>> > >
>>> > > 'main'
>>> > >
>>> > > >>
>>> > >
>>> > > >> to 2.0.
>>> > >
>>> > > >>
>>> > >
>>> > > >>
>>> > >
>>> > > >> Going forward then all work on the 1.x line should be focused on
>>> > >
>>> > > >>
>>> > >
>>> > > >> maintaining existing features, dependencies, and helping 1.x
>>> > > >> users
>>> > >
>>> > > migrate
>>> > >
>>> > > >>
>>> > >
>>> > > >> to the 2.x line. Otherwise, new feature work will happen on
>>> > > >> 'main' as
>>> > it
>>> > >
>>> > > >>
>>> > >
>>> > > >> normally does and will come out in the NiFi 2.x release line.
>>> > >
>>> > > >>
>>> > >
>>> > > >>
>>> > >
>>> > > >> Please flag key outstanding items as we narrow down the release
>>> > >
>>> > > candidate
>>> > >
>>> > > >>
>>> > >
>>> > > >> for NiFi 1.20.
>>> > >
>>> > > >>
>>> > >
>>> > > >>
>>> > >
>>> > > >> Thanks
>>> > >
>>> > > >>
>>> > >
>>> > > >> Joe
>>> > >
>>> > > >>
>>> > >
>>> > > >>
>>> > >
>>> > > >> [1]
>>> > > >> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2
>>> > > >> Flists.apache.org%2Fthread%2Fqo4vvdw46235y7vy2crcd6l4m11wl7jz&dat
>>> > > >> a=05%7C01%7Cisha.lamboo%40virtualsciences.nl%7Ccbea974a2c1f479d48
>>> > > >> 9d08daf42521f1%7C21429da9e4ad45f99a6fcd126a64274b%7C0%7C0%7C63809
>>> > > >> 0737572228694%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjo
>>> > > >>
>>> iV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=zSqOK9zZPqXxLuxwo0QcqKGEAc7aXjfnnm4i%2BQt2B98%3D&reserved=0
>>> > >
>>> > > >>
>>> > >
>>> > > >>
>>> > >
>>> > > >>
>>> > >
>>> > >
>>> > >
>>> >
>>>
>>
Re: [discuss] NiFi 1.20 and NiFi 2.0
Posted by Joe Witt <jo...@gmail.com>.
Hello
Here is a good picture of what the 1.20 RC looks like. I've tagged several
JIRAs today to ensure we get them in. A theme is really around stabilizing
nifi/nifi-registry integration as we're seeing substantial uptick in usage
and thus various community reported findings. We'll get that quite smooth
with these included.
https://issues.apache.org/jira/projects/NIFI/versions/12352581
Thanks
On Mon, Jan 23, 2023 at 8:50 AM Joe Witt <jo...@gmail.com> wrote:
> Team,
>
> I'm going through the outstanding JIRAs/PRs and flagging which look like
> they should be 'must have' for 1.20 and then will work the RC as soon as
> those land.
>
> Hopefully have the RC up within a day or two but we'll see how these land
> as some have review comments pending action.
>
> Thanks
>
> On Thu, Jan 12, 2023 at 2:53 AM Isha Lamboo <
> isha.lamboo@virtualsciences.nl> wrote:
>
>> Hi all,
>>
>> I would like to contribute to the migration tooling (mostly testing I
>> suppose) when that comes up.
>>
>> My team's largest client has a completely template-based pipeline with
>> external scripts replacing variable values before deploying to target
>> clusters, so we've already started looking at this when the goals for 2.0
>> were discussed and approved. The migration to flowdefinitions and
>> parameters is quite complex and we've hit several blockers when we tried to
>> implement a direct translation.
>>
>> I expect that any time I spend helping to improve the tooling will pay
>> off handsomely for our clients.
>>
>> Regards,
>>
>> Isha
>>
>> -----Oorspronkelijk bericht-----
>> Van: Adam Taft <ad...@adamtaft.com>
>> Verzonden: woensdag 11 januari 2023 23:42
>> Aan: dev@nifi.apache.org
>> Onderwerp: Re: [discuss] NiFi 1.20 and NiFi 2.0
>>
>> This is really insightful and spot on ...
>>
>> Kevin wrote:
>> > Good migration tooling will take a while to develop and test, and the
>> > core contributors to that effort may not have sufficient variety of
>> > flows to evaluate when the migration tools are "done" for the majority
>> > of the community to have success upgrading to 2.x. A milestone release
>> > would
>> allow
>> > us get more feedback on migration over a longer period than the vote
>> window
>> > of an RC candidate.
>>
>> It's exactly this case, that an early 2.0 release might not have had time
>> to fully work its way through existing production deployments, that's
>> concerning. The pace and voting of an "RC" is much too short to get any
>> quality feedback from users in the field.
>>
>> I think it's really smart to consider the "Milestone" release approach
>> here. We release 2.0.0-M1, 2.0.0-M2, ... waiting an adequate amount of time
>> for feedback. We can put these milestones on a calendar, as needed, so that
>> feedback is required some 'x' number of weeks/months after each milestone.
>>
>> And to this end, I'd personally rather see us keep the 'main' branch
>> current with the 1.x line _until_ we're ready and are satisfied with the
>> end goals of the 2.0 release objectives. When the milestone releases have
>> been completed and there's a comfort level with the 2.x line, it's at the
>> point we'd isolate the 1.x line into its own branch and switch main over to
>> the 2.x line.
>>
>> This is an attractive way of:
>> a) continuing business-as-usual with the 1.x line
>> b) making headway on the 2.x release milestones
>> c) giving adequate time for feedback against the 2.0 milestones coming
>> from the field
>>
>> I don't mind the known-unknowns. But it's really the unknown-unknowns
>> that are going to drive a delay in the 2.0 release. I think it's smart to
>> be able to get some of the unknowns ironed out before we finalize the 2.0
>> release ceremony. The milestone approach really helps with that.
>>
>> /Adam
>>
>> On Wed, Jan 11, 2023 at 11:11 AM Kevin Doran <kd...@apache.org> wrote:
>>
>> > Sorry, Joe, I was not clear, and to be honest the two thoughts are
>> > somewhat unrelated in my mind too :)
>> >
>> > I agree that good migration tooling is key. Otherwise, we risk users
>> > staying on 1.x or creating a schism of 1.x and 2.x users.
>> >
>> > Good migration tooling will take a while to develop and test, and the
>> > core contributors to that effort may not have sufficient variety of
>> > flows to evaluate when the migration tools are "done" for the majority
>> > of the community to have success upgrading to 2.x. A milestone release
>> > would allow us get more feedback on migration over a longer period
>> > than the vote window of an RC candidate.
>> >
>> > Perhaps we could continue to release from the 1.x line (including
>> > minor releases with some features) until we are ready to drop the
>> "milestone"
>> > qualifier from 2.0.0, and only then put 1.x into maintenance-only
>> status.
>> > It would be the same proposal to move main to target 2.0.0-M1, but
>> > relax restrictions for what can land on the 1.x branch and be open to
>> > a 1.21, 1.22, etc. if 2.0.0 work takes longer than anticipated. For
>> > example, maybe we would be open to landing new/backported processors
>> > on the 1.x branch, but not core framework features or API changes.
>> >
>> > This might not be necessary, but I think it is fair that saying "no
>> > new features on 1.x" and also "no new features in 2.0.0" puts the
>> > project in a rough place if 2.0.0 takes longer than a few months, so
>> > if we go that route, we need to commit to a quick release of 2.0.0
>> > that most users can move to easily.
>> >
>> > Thanks,
>> > Kevin
>> >
>> > On Jan 11, 2023 at 12:32:46, Joe Witt <jo...@gmail.com> wrote:
>> >
>> > > Kevin,
>> > >
>> > > Yeah we can do whatever we want as far as 'releases' of 2.0 that are
>> > prior
>> > > to us officially considering it 2.0/stable.
>> > >
>> > > That said - the migration tooling will be key to provide as we need
>> > > to
>> > make
>> > > the bridge as solid and stable as possible to help someone move from
>> > > 1.x
>> > to
>> > > 2.x. I dont know how related these two concepts (milestone releases
>> > > and 1.x to 2.x ease really are).
>> > >
>> > > Thanks
>> > >
>> > > On Wed, Jan 11, 2023 at 10:27 AM Kevin Doran <kd...@apache.org>
>> wrote:
>> > >
>> > > [hit the wrong keyboard shortcut, here is the rest of my thoughts]
>> > >
>> > >
>> > > On this point from David:
>> > >
>> > >
>> > > We may need to have a longer release candidate period, or more
>> > incremental
>> > >
>> > > > fix releases
>> > >
>> > > > for the initial 2.0.0 release train, but I do not expect delaying
>> > > > a
>> > 2.0.0
>> > >
>> > > > release for new features, as that is not part of the release goals.
>> > >
>> > > >
>> > >
>> > >
>> > > Would the community benefit from one or more milestone releases of
>> > > 2.0,
>> > to
>> > >
>> > > allow for a wider group to run / live on the proposed 2.0 prior to
>> > >
>> > > releasing it as "stable"? I know we've never done a milestone
>> > > release in
>> > >
>> > > the past, and I'm not sure what ASF guidance is on the topic, but if
>> > > it
>> > >
>> > > could be beneficial we could look into that.
>> > >
>> > >
>> > > Cheers,
>> > >
>> > > Kevin
>> > >
>> > >
>> > > On Jan 11, 2023 at 12:22:43, Kevin Doran <kd...@apache.org> wrote:
>> > >
>> > >
>> > > > I think this is a good, practical discussion.
>> > >
>> > > >
>> > >
>> > > > On the one hand, we can't put off 2.x any longer as we really need
>> > > > to
>> > >
>> > > > updated the minimum required Java to 11. Updating main development
>> > > > to
>> > >
>> > > > target 2.x feels like a good way drive progress on that along with
>> > > > the
>> > >
>> > > > other 2.0 goals.
>> > >
>> > > >
>> > >
>> > > > On the other hand, the concerns are valid: moving all development
>> > > > to
>> > >
>> > > > target 2.x puts the project at risk if we cannot release 2.0.0 on
>> > > > a
>> > >
>> > > > reasonable timeline. The restricted scope of 2.0 helps, but this
>> > > > stated
>> > >
>> > > > release goal feels risky to me:
>> > >
>> > > >
>> > >
>> > > > Implement Migration Tools for Upgrading Flows
>> > >
>> > > >
>> > >
>> > > >
>> > >
>> > > > - Implement automated migration where possible to remap
>> > > > properties
>> > and
>> > >
>> > > > features
>> > >
>> > > > - Implement migration tools for manual conversion of XML
>> > Templates
>> > >
>> > > > to JSON Flow Definitions
>> > >
>> > > > - Create documentation for manual steps necessary where
>> > >
>> > > > programmatic migration cannot be implemented
>> > >
>> > > > - NiFi 2.0 should be capable of starting with ghosted
>> > > > components
>> > >
>> > > > for removed Processors or Controller Services
>> > >
>> > > >
>> > >
>> > > >
>> > >
>> > > > Removing deprecated components should be fairly straightforward
>> > > > and
>> > >
>> > > quick,
>> > >
>> > > > but automating and documenting migration is a large effort.
>> > >
>> > > >
>> > >
>> > > > On this po
>> > >
>> > > >
>> > >
>> > > >
>> > >
>> > > > On Jan 10, 2023 at 09:32:31, Bryan Bende <bb...@gmail.com> wrote:
>> > >
>> > > >
>> > >
>> > > >> The plan as I understand it is not to diverge and create separate
>> > >
>> > > feature
>> > >
>> > > >> development on the 1.x line, so I would expect all PRs to
>> > > >> continue to
>> > be
>> > >
>> > > >> submitted only to main. We would release 1.x as needed with major
>> > > >> bug
>> > >
>> > > >> fixes
>> > >
>> > > >> or critical security updates, and these would be cherry-picked
>> > > >> and/or
>> > >
>> > > >> backported as necessary, mostly without the need for PRs, the
>> > > >> same as
>> > we
>> > >
>> > > >> would do if we were bringing fixes from main (1.20.0-SNAPSHOT)
>> > > >> back
>> > to a
>> > >
>> > > >> maintenance line like (1.19.x). For precedent, we followed this
>> > > >> same
>> > >
>> > > >> approach going from the 0.x line to 1.0.0 and there wasn't any
>> > > >> major
>> > >
>> > > >> issue.
>> > >
>> > > >>
>> > >
>> > > >> On Tue, Jan 10, 2023 at 7:07 AM Otto Fowler
>> > > >> <ot...@gmail.com>
>> > >
>> > > >> wrote:
>> > >
>> > > >>
>> > >
>> > > >> It was also mentioned in another thread that we need to have
>> > agreement
>> > >
>> > > on
>> > >
>> > > >>
>> > >
>> > > >> our explicit strategy and support for 1.x going forward, did that
>> > >
>> > > happen?
>> > >
>> > > >>
>> > >
>> > > >>
>> > >
>> > > >> From: Otto Fowler <ot...@gmail.com>
>> > > >> <ot...@gmail.com>
>> > >
>> > > >>
>> > >
>> > > >> Reply: Otto Fowler <ot...@gmail.com>
>> > > >> <ottobackwards@gmail.com
>> > >
>> > >
>> > > >>
>> > >
>> > > >> Date: January 10, 2023 at 07:02:34
>> > >
>> > > >>
>> > >
>> > > >> To: dev@nifi.apache.org <de...@nifi.apache.org>
>> > > >> <de...@nifi.apache.org>
>> > >
>> > > >>
>> > >
>> > > >> Subject: Re: [discuss] NiFi 1.20 and NiFi 2.0
>> > >
>> > > >>
>> > >
>> > > >>
>> > >
>> > > >> There needs to be an update to the contributing guide as to how
>> > > >> to
>> > >
>> > > submit
>> > >
>> > > >>
>> > >
>> > > >> PRs to 1.x or 2.x etc.
>> > >
>> > > >>
>> > >
>> > > >>
>> > >
>> > > >> From: Joe Witt <jo...@apache.org> <jo...@apache.org>
>> > >
>> > > >>
>> > >
>> > > >> Reply: dev@nifi.apache.org <de...@nifi.apache.org>
>> > > >> <dev@nifi.apache.org
>> > >
>> > >
>> > > >>
>> > >
>> > > >> Date: January 9, 2023 at 15:53:16
>> > >
>> > > >>
>> > >
>> > > >> To: dev@nifi.apache.org <de...@nifi.apache.org>
>> > > >> <de...@nifi.apache.org>
>> > >
>> > > >>
>> > >
>> > > >> Subject: [discuss] NiFi 1.20 and NiFi 2.0
>> > >
>> > > >>
>> > >
>> > > >>
>> > >
>> > > >> Team,
>> > >
>> > > >>
>> > >
>> > > >>
>> > >
>> > > >> As David mentioned in [1] following a successful NiFi 2.0 release
>> > > >> goal
>> > >
>> > > >>
>> > >
>> > > >> planning - we are now going to start moving the 'main' line to be
>> > > >> the
>> > >
>> > > NiFi
>> > >
>> > > >>
>> > >
>> > > >> 2.0 line which will allow for the key work to take place. We will
>> > > >> also
>> > >
>> > > >>
>> > >
>> > > >> move niFi 1.x to its appropriate support line.
>> > >
>> > > >>
>> > >
>> > > >>
>> > >
>> > > >> It is also the case that we have nearly 100 JIRAs on NiFi 1.20
>> > > >> and we
>> > >
>> > > have
>> > >
>> > > >>
>> > >
>> > > >> work in there including security items so it is time to make a
>> > release.
>> > >
>> > > >>
>> > >
>> > > >> The intent then is to initiate 1.20 and immediate after that
>> > > >> change
>> > >
>> > > 'main'
>> > >
>> > > >>
>> > >
>> > > >> to 2.0.
>> > >
>> > > >>
>> > >
>> > > >>
>> > >
>> > > >> Going forward then all work on the 1.x line should be focused on
>> > >
>> > > >>
>> > >
>> > > >> maintaining existing features, dependencies, and helping 1.x
>> > > >> users
>> > >
>> > > migrate
>> > >
>> > > >>
>> > >
>> > > >> to the 2.x line. Otherwise, new feature work will happen on
>> > > >> 'main' as
>> > it
>> > >
>> > > >>
>> > >
>> > > >> normally does and will come out in the NiFi 2.x release line.
>> > >
>> > > >>
>> > >
>> > > >>
>> > >
>> > > >> Please flag key outstanding items as we narrow down the release
>> > >
>> > > candidate
>> > >
>> > > >>
>> > >
>> > > >> for NiFi 1.20.
>> > >
>> > > >>
>> > >
>> > > >>
>> > >
>> > > >> Thanks
>> > >
>> > > >>
>> > >
>> > > >> Joe
>> > >
>> > > >>
>> > >
>> > > >>
>> > >
>> > > >> [1]
>> > > >> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2
>> > > >> Flists.apache.org%2Fthread%2Fqo4vvdw46235y7vy2crcd6l4m11wl7jz&dat
>> > > >> a=05%7C01%7Cisha.lamboo%40virtualsciences.nl%7Ccbea974a2c1f479d48
>> > > >> 9d08daf42521f1%7C21429da9e4ad45f99a6fcd126a64274b%7C0%7C0%7C63809
>> > > >> 0737572228694%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjo
>> > > >>
>> iV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=zSqOK9zZPqXxLuxwo0QcqKGEAc7aXjfnnm4i%2BQt2B98%3D&reserved=0
>> > >
>> > > >>
>> > >
>> > > >>
>> > >
>> > > >>
>> > >
>> > >
>> > >
>> >
>>
>
Re: [discuss] NiFi 1.20 and NiFi 2.0
Posted by Joe Witt <jo...@gmail.com>.
Team,
I'm going through the outstanding JIRAs/PRs and flagging which look like
they should be 'must have' for 1.20 and then will work the RC as soon as
those land.
Hopefully have the RC up within a day or two but we'll see how these land
as some have review comments pending action.
Thanks
On Thu, Jan 12, 2023 at 2:53 AM Isha Lamboo <is...@virtualsciences.nl>
wrote:
> Hi all,
>
> I would like to contribute to the migration tooling (mostly testing I
> suppose) when that comes up.
>
> My team's largest client has a completely template-based pipeline with
> external scripts replacing variable values before deploying to target
> clusters, so we've already started looking at this when the goals for 2.0
> were discussed and approved. The migration to flowdefinitions and
> parameters is quite complex and we've hit several blockers when we tried to
> implement a direct translation.
>
> I expect that any time I spend helping to improve the tooling will pay off
> handsomely for our clients.
>
> Regards,
>
> Isha
>
> -----Oorspronkelijk bericht-----
> Van: Adam Taft <ad...@adamtaft.com>
> Verzonden: woensdag 11 januari 2023 23:42
> Aan: dev@nifi.apache.org
> Onderwerp: Re: [discuss] NiFi 1.20 and NiFi 2.0
>
> This is really insightful and spot on ...
>
> Kevin wrote:
> > Good migration tooling will take a while to develop and test, and the
> > core contributors to that effort may not have sufficient variety of
> > flows to evaluate when the migration tools are "done" for the majority
> > of the community to have success upgrading to 2.x. A milestone release
> > would
> allow
> > us get more feedback on migration over a longer period than the vote
> window
> > of an RC candidate.
>
> It's exactly this case, that an early 2.0 release might not have had time
> to fully work its way through existing production deployments, that's
> concerning. The pace and voting of an "RC" is much too short to get any
> quality feedback from users in the field.
>
> I think it's really smart to consider the "Milestone" release approach
> here. We release 2.0.0-M1, 2.0.0-M2, ... waiting an adequate amount of time
> for feedback. We can put these milestones on a calendar, as needed, so that
> feedback is required some 'x' number of weeks/months after each milestone.
>
> And to this end, I'd personally rather see us keep the 'main' branch
> current with the 1.x line _until_ we're ready and are satisfied with the
> end goals of the 2.0 release objectives. When the milestone releases have
> been completed and there's a comfort level with the 2.x line, it's at the
> point we'd isolate the 1.x line into its own branch and switch main over to
> the 2.x line.
>
> This is an attractive way of:
> a) continuing business-as-usual with the 1.x line
> b) making headway on the 2.x release milestones
> c) giving adequate time for feedback against the 2.0 milestones coming
> from the field
>
> I don't mind the known-unknowns. But it's really the unknown-unknowns that
> are going to drive a delay in the 2.0 release. I think it's smart to be
> able to get some of the unknowns ironed out before we finalize the 2.0
> release ceremony. The milestone approach really helps with that.
>
> /Adam
>
> On Wed, Jan 11, 2023 at 11:11 AM Kevin Doran <kd...@apache.org> wrote:
>
> > Sorry, Joe, I was not clear, and to be honest the two thoughts are
> > somewhat unrelated in my mind too :)
> >
> > I agree that good migration tooling is key. Otherwise, we risk users
> > staying on 1.x or creating a schism of 1.x and 2.x users.
> >
> > Good migration tooling will take a while to develop and test, and the
> > core contributors to that effort may not have sufficient variety of
> > flows to evaluate when the migration tools are "done" for the majority
> > of the community to have success upgrading to 2.x. A milestone release
> > would allow us get more feedback on migration over a longer period
> > than the vote window of an RC candidate.
> >
> > Perhaps we could continue to release from the 1.x line (including
> > minor releases with some features) until we are ready to drop the
> "milestone"
> > qualifier from 2.0.0, and only then put 1.x into maintenance-only status.
> > It would be the same proposal to move main to target 2.0.0-M1, but
> > relax restrictions for what can land on the 1.x branch and be open to
> > a 1.21, 1.22, etc. if 2.0.0 work takes longer than anticipated. For
> > example, maybe we would be open to landing new/backported processors
> > on the 1.x branch, but not core framework features or API changes.
> >
> > This might not be necessary, but I think it is fair that saying "no
> > new features on 1.x" and also "no new features in 2.0.0" puts the
> > project in a rough place if 2.0.0 takes longer than a few months, so
> > if we go that route, we need to commit to a quick release of 2.0.0
> > that most users can move to easily.
> >
> > Thanks,
> > Kevin
> >
> > On Jan 11, 2023 at 12:32:46, Joe Witt <jo...@gmail.com> wrote:
> >
> > > Kevin,
> > >
> > > Yeah we can do whatever we want as far as 'releases' of 2.0 that are
> > prior
> > > to us officially considering it 2.0/stable.
> > >
> > > That said - the migration tooling will be key to provide as we need
> > > to
> > make
> > > the bridge as solid and stable as possible to help someone move from
> > > 1.x
> > to
> > > 2.x. I dont know how related these two concepts (milestone releases
> > > and 1.x to 2.x ease really are).
> > >
> > > Thanks
> > >
> > > On Wed, Jan 11, 2023 at 10:27 AM Kevin Doran <kd...@apache.org>
> wrote:
> > >
> > > [hit the wrong keyboard shortcut, here is the rest of my thoughts]
> > >
> > >
> > > On this point from David:
> > >
> > >
> > > We may need to have a longer release candidate period, or more
> > incremental
> > >
> > > > fix releases
> > >
> > > > for the initial 2.0.0 release train, but I do not expect delaying
> > > > a
> > 2.0.0
> > >
> > > > release for new features, as that is not part of the release goals.
> > >
> > > >
> > >
> > >
> > > Would the community benefit from one or more milestone releases of
> > > 2.0,
> > to
> > >
> > > allow for a wider group to run / live on the proposed 2.0 prior to
> > >
> > > releasing it as "stable"? I know we've never done a milestone
> > > release in
> > >
> > > the past, and I'm not sure what ASF guidance is on the topic, but if
> > > it
> > >
> > > could be beneficial we could look into that.
> > >
> > >
> > > Cheers,
> > >
> > > Kevin
> > >
> > >
> > > On Jan 11, 2023 at 12:22:43, Kevin Doran <kd...@apache.org> wrote:
> > >
> > >
> > > > I think this is a good, practical discussion.
> > >
> > > >
> > >
> > > > On the one hand, we can't put off 2.x any longer as we really need
> > > > to
> > >
> > > > updated the minimum required Java to 11. Updating main development
> > > > to
> > >
> > > > target 2.x feels like a good way drive progress on that along with
> > > > the
> > >
> > > > other 2.0 goals.
> > >
> > > >
> > >
> > > > On the other hand, the concerns are valid: moving all development
> > > > to
> > >
> > > > target 2.x puts the project at risk if we cannot release 2.0.0 on
> > > > a
> > >
> > > > reasonable timeline. The restricted scope of 2.0 helps, but this
> > > > stated
> > >
> > > > release goal feels risky to me:
> > >
> > > >
> > >
> > > > Implement Migration Tools for Upgrading Flows
> > >
> > > >
> > >
> > > >
> > >
> > > > - Implement automated migration where possible to remap
> > > > properties
> > and
> > >
> > > > features
> > >
> > > > - Implement migration tools for manual conversion of XML
> > Templates
> > >
> > > > to JSON Flow Definitions
> > >
> > > > - Create documentation for manual steps necessary where
> > >
> > > > programmatic migration cannot be implemented
> > >
> > > > - NiFi 2.0 should be capable of starting with ghosted
> > > > components
> > >
> > > > for removed Processors or Controller Services
> > >
> > > >
> > >
> > > >
> > >
> > > > Removing deprecated components should be fairly straightforward
> > > > and
> > >
> > > quick,
> > >
> > > > but automating and documenting migration is a large effort.
> > >
> > > >
> > >
> > > > On this po
> > >
> > > >
> > >
> > > >
> > >
> > > > On Jan 10, 2023 at 09:32:31, Bryan Bende <bb...@gmail.com> wrote:
> > >
> > > >
> > >
> > > >> The plan as I understand it is not to diverge and create separate
> > >
> > > feature
> > >
> > > >> development on the 1.x line, so I would expect all PRs to
> > > >> continue to
> > be
> > >
> > > >> submitted only to main. We would release 1.x as needed with major
> > > >> bug
> > >
> > > >> fixes
> > >
> > > >> or critical security updates, and these would be cherry-picked
> > > >> and/or
> > >
> > > >> backported as necessary, mostly without the need for PRs, the
> > > >> same as
> > we
> > >
> > > >> would do if we were bringing fixes from main (1.20.0-SNAPSHOT)
> > > >> back
> > to a
> > >
> > > >> maintenance line like (1.19.x). For precedent, we followed this
> > > >> same
> > >
> > > >> approach going from the 0.x line to 1.0.0 and there wasn't any
> > > >> major
> > >
> > > >> issue.
> > >
> > > >>
> > >
> > > >> On Tue, Jan 10, 2023 at 7:07 AM Otto Fowler
> > > >> <ot...@gmail.com>
> > >
> > > >> wrote:
> > >
> > > >>
> > >
> > > >> It was also mentioned in another thread that we need to have
> > agreement
> > >
> > > on
> > >
> > > >>
> > >
> > > >> our explicit strategy and support for 1.x going forward, did that
> > >
> > > happen?
> > >
> > > >>
> > >
> > > >>
> > >
> > > >> From: Otto Fowler <ot...@gmail.com>
> > > >> <ot...@gmail.com>
> > >
> > > >>
> > >
> > > >> Reply: Otto Fowler <ot...@gmail.com>
> > > >> <ottobackwards@gmail.com
> > >
> > >
> > > >>
> > >
> > > >> Date: January 10, 2023 at 07:02:34
> > >
> > > >>
> > >
> > > >> To: dev@nifi.apache.org <de...@nifi.apache.org>
> > > >> <de...@nifi.apache.org>
> > >
> > > >>
> > >
> > > >> Subject: Re: [discuss] NiFi 1.20 and NiFi 2.0
> > >
> > > >>
> > >
> > > >>
> > >
> > > >> There needs to be an update to the contributing guide as to how
> > > >> to
> > >
> > > submit
> > >
> > > >>
> > >
> > > >> PRs to 1.x or 2.x etc.
> > >
> > > >>
> > >
> > > >>
> > >
> > > >> From: Joe Witt <jo...@apache.org> <jo...@apache.org>
> > >
> > > >>
> > >
> > > >> Reply: dev@nifi.apache.org <de...@nifi.apache.org>
> > > >> <dev@nifi.apache.org
> > >
> > >
> > > >>
> > >
> > > >> Date: January 9, 2023 at 15:53:16
> > >
> > > >>
> > >
> > > >> To: dev@nifi.apache.org <de...@nifi.apache.org>
> > > >> <de...@nifi.apache.org>
> > >
> > > >>
> > >
> > > >> Subject: [discuss] NiFi 1.20 and NiFi 2.0
> > >
> > > >>
> > >
> > > >>
> > >
> > > >> Team,
> > >
> > > >>
> > >
> > > >>
> > >
> > > >> As David mentioned in [1] following a successful NiFi 2.0 release
> > > >> goal
> > >
> > > >>
> > >
> > > >> planning - we are now going to start moving the 'main' line to be
> > > >> the
> > >
> > > NiFi
> > >
> > > >>
> > >
> > > >> 2.0 line which will allow for the key work to take place. We will
> > > >> also
> > >
> > > >>
> > >
> > > >> move niFi 1.x to its appropriate support line.
> > >
> > > >>
> > >
> > > >>
> > >
> > > >> It is also the case that we have nearly 100 JIRAs on NiFi 1.20
> > > >> and we
> > >
> > > have
> > >
> > > >>
> > >
> > > >> work in there including security items so it is time to make a
> > release.
> > >
> > > >>
> > >
> > > >> The intent then is to initiate 1.20 and immediate after that
> > > >> change
> > >
> > > 'main'
> > >
> > > >>
> > >
> > > >> to 2.0.
> > >
> > > >>
> > >
> > > >>
> > >
> > > >> Going forward then all work on the 1.x line should be focused on
> > >
> > > >>
> > >
> > > >> maintaining existing features, dependencies, and helping 1.x
> > > >> users
> > >
> > > migrate
> > >
> > > >>
> > >
> > > >> to the 2.x line. Otherwise, new feature work will happen on
> > > >> 'main' as
> > it
> > >
> > > >>
> > >
> > > >> normally does and will come out in the NiFi 2.x release line.
> > >
> > > >>
> > >
> > > >>
> > >
> > > >> Please flag key outstanding items as we narrow down the release
> > >
> > > candidate
> > >
> > > >>
> > >
> > > >> for NiFi 1.20.
> > >
> > > >>
> > >
> > > >>
> > >
> > > >> Thanks
> > >
> > > >>
> > >
> > > >> Joe
> > >
> > > >>
> > >
> > > >>
> > >
> > > >> [1]
> > > >> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2
> > > >> Flists.apache.org%2Fthread%2Fqo4vvdw46235y7vy2crcd6l4m11wl7jz&dat
> > > >> a=05%7C01%7Cisha.lamboo%40virtualsciences.nl%7Ccbea974a2c1f479d48
> > > >> 9d08daf42521f1%7C21429da9e4ad45f99a6fcd126a64274b%7C0%7C0%7C63809
> > > >> 0737572228694%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjo
> > > >>
> iV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=zSqOK9zZPqXxLuxwo0QcqKGEAc7aXjfnnm4i%2BQt2B98%3D&reserved=0
> > >
> > > >>
> > >
> > > >>
> > >
> > > >>
> > >
> > >
> > >
> >
>
RE: [discuss] NiFi 1.20 and NiFi 2.0
Posted by Isha Lamboo <is...@virtualsciences.nl>.
Hi all,
I would like to contribute to the migration tooling (mostly testing I suppose) when that comes up.
My team's largest client has a completely template-based pipeline with external scripts replacing variable values before deploying to target clusters, so we've already started looking at this when the goals for 2.0 were discussed and approved. The migration to flowdefinitions and parameters is quite complex and we've hit several blockers when we tried to implement a direct translation.
I expect that any time I spend helping to improve the tooling will pay off handsomely for our clients.
Regards,
Isha
-----Oorspronkelijk bericht-----
Van: Adam Taft <ad...@adamtaft.com>
Verzonden: woensdag 11 januari 2023 23:42
Aan: dev@nifi.apache.org
Onderwerp: Re: [discuss] NiFi 1.20 and NiFi 2.0
This is really insightful and spot on ...
Kevin wrote:
> Good migration tooling will take a while to develop and test, and the
> core contributors to that effort may not have sufficient variety of
> flows to evaluate when the migration tools are "done" for the majority
> of the community to have success upgrading to 2.x. A milestone release
> would
allow
> us get more feedback on migration over a longer period than the vote
window
> of an RC candidate.
It's exactly this case, that an early 2.0 release might not have had time to fully work its way through existing production deployments, that's concerning. The pace and voting of an "RC" is much too short to get any quality feedback from users in the field.
I think it's really smart to consider the "Milestone" release approach here. We release 2.0.0-M1, 2.0.0-M2, ... waiting an adequate amount of time for feedback. We can put these milestones on a calendar, as needed, so that feedback is required some 'x' number of weeks/months after each milestone.
And to this end, I'd personally rather see us keep the 'main' branch current with the 1.x line _until_ we're ready and are satisfied with the end goals of the 2.0 release objectives. When the milestone releases have been completed and there's a comfort level with the 2.x line, it's at the point we'd isolate the 1.x line into its own branch and switch main over to the 2.x line.
This is an attractive way of:
a) continuing business-as-usual with the 1.x line
b) making headway on the 2.x release milestones
c) giving adequate time for feedback against the 2.0 milestones coming from the field
I don't mind the known-unknowns. But it's really the unknown-unknowns that are going to drive a delay in the 2.0 release. I think it's smart to be able to get some of the unknowns ironed out before we finalize the 2.0 release ceremony. The milestone approach really helps with that.
/Adam
On Wed, Jan 11, 2023 at 11:11 AM Kevin Doran <kd...@apache.org> wrote:
> Sorry, Joe, I was not clear, and to be honest the two thoughts are
> somewhat unrelated in my mind too :)
>
> I agree that good migration tooling is key. Otherwise, we risk users
> staying on 1.x or creating a schism of 1.x and 2.x users.
>
> Good migration tooling will take a while to develop and test, and the
> core contributors to that effort may not have sufficient variety of
> flows to evaluate when the migration tools are "done" for the majority
> of the community to have success upgrading to 2.x. A milestone release
> would allow us get more feedback on migration over a longer period
> than the vote window of an RC candidate.
>
> Perhaps we could continue to release from the 1.x line (including
> minor releases with some features) until we are ready to drop the "milestone"
> qualifier from 2.0.0, and only then put 1.x into maintenance-only status.
> It would be the same proposal to move main to target 2.0.0-M1, but
> relax restrictions for what can land on the 1.x branch and be open to
> a 1.21, 1.22, etc. if 2.0.0 work takes longer than anticipated. For
> example, maybe we would be open to landing new/backported processors
> on the 1.x branch, but not core framework features or API changes.
>
> This might not be necessary, but I think it is fair that saying "no
> new features on 1.x" and also "no new features in 2.0.0" puts the
> project in a rough place if 2.0.0 takes longer than a few months, so
> if we go that route, we need to commit to a quick release of 2.0.0
> that most users can move to easily.
>
> Thanks,
> Kevin
>
> On Jan 11, 2023 at 12:32:46, Joe Witt <jo...@gmail.com> wrote:
>
> > Kevin,
> >
> > Yeah we can do whatever we want as far as 'releases' of 2.0 that are
> prior
> > to us officially considering it 2.0/stable.
> >
> > That said - the migration tooling will be key to provide as we need
> > to
> make
> > the bridge as solid and stable as possible to help someone move from
> > 1.x
> to
> > 2.x. I dont know how related these two concepts (milestone releases
> > and 1.x to 2.x ease really are).
> >
> > Thanks
> >
> > On Wed, Jan 11, 2023 at 10:27 AM Kevin Doran <kd...@apache.org> wrote:
> >
> > [hit the wrong keyboard shortcut, here is the rest of my thoughts]
> >
> >
> > On this point from David:
> >
> >
> > We may need to have a longer release candidate period, or more
> incremental
> >
> > > fix releases
> >
> > > for the initial 2.0.0 release train, but I do not expect delaying
> > > a
> 2.0.0
> >
> > > release for new features, as that is not part of the release goals.
> >
> > >
> >
> >
> > Would the community benefit from one or more milestone releases of
> > 2.0,
> to
> >
> > allow for a wider group to run / live on the proposed 2.0 prior to
> >
> > releasing it as "stable"? I know we've never done a milestone
> > release in
> >
> > the past, and I'm not sure what ASF guidance is on the topic, but if
> > it
> >
> > could be beneficial we could look into that.
> >
> >
> > Cheers,
> >
> > Kevin
> >
> >
> > On Jan 11, 2023 at 12:22:43, Kevin Doran <kd...@apache.org> wrote:
> >
> >
> > > I think this is a good, practical discussion.
> >
> > >
> >
> > > On the one hand, we can't put off 2.x any longer as we really need
> > > to
> >
> > > updated the minimum required Java to 11. Updating main development
> > > to
> >
> > > target 2.x feels like a good way drive progress on that along with
> > > the
> >
> > > other 2.0 goals.
> >
> > >
> >
> > > On the other hand, the concerns are valid: moving all development
> > > to
> >
> > > target 2.x puts the project at risk if we cannot release 2.0.0 on
> > > a
> >
> > > reasonable timeline. The restricted scope of 2.0 helps, but this
> > > stated
> >
> > > release goal feels risky to me:
> >
> > >
> >
> > > Implement Migration Tools for Upgrading Flows
> >
> > >
> >
> > >
> >
> > > - Implement automated migration where possible to remap
> > > properties
> and
> >
> > > features
> >
> > > - Implement migration tools for manual conversion of XML
> Templates
> >
> > > to JSON Flow Definitions
> >
> > > - Create documentation for manual steps necessary where
> >
> > > programmatic migration cannot be implemented
> >
> > > - NiFi 2.0 should be capable of starting with ghosted
> > > components
> >
> > > for removed Processors or Controller Services
> >
> > >
> >
> > >
> >
> > > Removing deprecated components should be fairly straightforward
> > > and
> >
> > quick,
> >
> > > but automating and documenting migration is a large effort.
> >
> > >
> >
> > > On this po
> >
> > >
> >
> > >
> >
> > > On Jan 10, 2023 at 09:32:31, Bryan Bende <bb...@gmail.com> wrote:
> >
> > >
> >
> > >> The plan as I understand it is not to diverge and create separate
> >
> > feature
> >
> > >> development on the 1.x line, so I would expect all PRs to
> > >> continue to
> be
> >
> > >> submitted only to main. We would release 1.x as needed with major
> > >> bug
> >
> > >> fixes
> >
> > >> or critical security updates, and these would be cherry-picked
> > >> and/or
> >
> > >> backported as necessary, mostly without the need for PRs, the
> > >> same as
> we
> >
> > >> would do if we were bringing fixes from main (1.20.0-SNAPSHOT)
> > >> back
> to a
> >
> > >> maintenance line like (1.19.x). For precedent, we followed this
> > >> same
> >
> > >> approach going from the 0.x line to 1.0.0 and there wasn't any
> > >> major
> >
> > >> issue.
> >
> > >>
> >
> > >> On Tue, Jan 10, 2023 at 7:07 AM Otto Fowler
> > >> <ot...@gmail.com>
> >
> > >> wrote:
> >
> > >>
> >
> > >> It was also mentioned in another thread that we need to have
> agreement
> >
> > on
> >
> > >>
> >
> > >> our explicit strategy and support for 1.x going forward, did that
> >
> > happen?
> >
> > >>
> >
> > >>
> >
> > >> From: Otto Fowler <ot...@gmail.com>
> > >> <ot...@gmail.com>
> >
> > >>
> >
> > >> Reply: Otto Fowler <ot...@gmail.com>
> > >> <ottobackwards@gmail.com
> >
> >
> > >>
> >
> > >> Date: January 10, 2023 at 07:02:34
> >
> > >>
> >
> > >> To: dev@nifi.apache.org <de...@nifi.apache.org>
> > >> <de...@nifi.apache.org>
> >
> > >>
> >
> > >> Subject: Re: [discuss] NiFi 1.20 and NiFi 2.0
> >
> > >>
> >
> > >>
> >
> > >> There needs to be an update to the contributing guide as to how
> > >> to
> >
> > submit
> >
> > >>
> >
> > >> PRs to 1.x or 2.x etc.
> >
> > >>
> >
> > >>
> >
> > >> From: Joe Witt <jo...@apache.org> <jo...@apache.org>
> >
> > >>
> >
> > >> Reply: dev@nifi.apache.org <de...@nifi.apache.org>
> > >> <dev@nifi.apache.org
> >
> >
> > >>
> >
> > >> Date: January 9, 2023 at 15:53:16
> >
> > >>
> >
> > >> To: dev@nifi.apache.org <de...@nifi.apache.org>
> > >> <de...@nifi.apache.org>
> >
> > >>
> >
> > >> Subject: [discuss] NiFi 1.20 and NiFi 2.0
> >
> > >>
> >
> > >>
> >
> > >> Team,
> >
> > >>
> >
> > >>
> >
> > >> As David mentioned in [1] following a successful NiFi 2.0 release
> > >> goal
> >
> > >>
> >
> > >> planning - we are now going to start moving the 'main' line to be
> > >> the
> >
> > NiFi
> >
> > >>
> >
> > >> 2.0 line which will allow for the key work to take place. We will
> > >> also
> >
> > >>
> >
> > >> move niFi 1.x to its appropriate support line.
> >
> > >>
> >
> > >>
> >
> > >> It is also the case that we have nearly 100 JIRAs on NiFi 1.20
> > >> and we
> >
> > have
> >
> > >>
> >
> > >> work in there including security items so it is time to make a
> release.
> >
> > >>
> >
> > >> The intent then is to initiate 1.20 and immediate after that
> > >> change
> >
> > 'main'
> >
> > >>
> >
> > >> to 2.0.
> >
> > >>
> >
> > >>
> >
> > >> Going forward then all work on the 1.x line should be focused on
> >
> > >>
> >
> > >> maintaining existing features, dependencies, and helping 1.x
> > >> users
> >
> > migrate
> >
> > >>
> >
> > >> to the 2.x line. Otherwise, new feature work will happen on
> > >> 'main' as
> it
> >
> > >>
> >
> > >> normally does and will come out in the NiFi 2.x release line.
> >
> > >>
> >
> > >>
> >
> > >> Please flag key outstanding items as we narrow down the release
> >
> > candidate
> >
> > >>
> >
> > >> for NiFi 1.20.
> >
> > >>
> >
> > >>
> >
> > >> Thanks
> >
> > >>
> >
> > >> Joe
> >
> > >>
> >
> > >>
> >
> > >> [1]
> > >> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2
> > >> Flists.apache.org%2Fthread%2Fqo4vvdw46235y7vy2crcd6l4m11wl7jz&dat
> > >> a=05%7C01%7Cisha.lamboo%40virtualsciences.nl%7Ccbea974a2c1f479d48
> > >> 9d08daf42521f1%7C21429da9e4ad45f99a6fcd126a64274b%7C0%7C0%7C63809
> > >> 0737572228694%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjo
> > >> iV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=zSqOK9zZPqXxLuxwo0QcqKGEAc7aXjfnnm4i%2BQt2B98%3D&reserved=0
> >
> > >>
> >
> > >>
> >
> > >>
> >
> >
> >
>
Re: [discuss] NiFi 1.20 and NiFi 2.0
Posted by Adam Taft <ad...@adamtaft.com>.
This is really insightful and spot on ...
Kevin wrote:
> Good migration tooling will take a while to develop and test, and the core
> contributors to that effort may not have sufficient variety of flows to
> evaluate when the migration tools are "done" for the majority of the
> community to have success upgrading to 2.x. A milestone release would
allow
> us get more feedback on migration over a longer period than the vote
window
> of an RC candidate.
It's exactly this case, that an early 2.0 release might not have had time
to fully work its way through existing production deployments, that's
concerning. The pace and voting of an "RC" is much too short to get any
quality feedback from users in the field.
I think it's really smart to consider the "Milestone" release approach
here. We release 2.0.0-M1, 2.0.0-M2, ... waiting an adequate amount of time
for feedback. We can put these milestones on a calendar, as needed, so that
feedback is required some 'x' number of weeks/months after each milestone.
And to this end, I'd personally rather see us keep the 'main' branch
current with the 1.x line _until_ we're ready and are satisfied with the
end goals of the 2.0 release objectives. When the milestone releases have
been completed and there's a comfort level with the 2.x line, it's at the
point we'd isolate the 1.x line into its own branch and switch main over to
the 2.x line.
This is an attractive way of:
a) continuing business-as-usual with the 1.x line
b) making headway on the 2.x release milestones
c) giving adequate time for feedback against the 2.0 milestones coming from
the field
I don't mind the known-unknowns. But it's really the unknown-unknowns that
are going to drive a delay in the 2.0 release. I think it's smart to be
able to get some of the unknowns ironed out before we finalize the 2.0
release ceremony. The milestone approach really helps with that.
/Adam
On Wed, Jan 11, 2023 at 11:11 AM Kevin Doran <kd...@apache.org> wrote:
> Sorry, Joe, I was not clear, and to be honest the two thoughts are somewhat
> unrelated in my mind too :)
>
> I agree that good migration tooling is key. Otherwise, we risk users
> staying on 1.x or creating a schism of 1.x and 2.x users.
>
> Good migration tooling will take a while to develop and test, and the core
> contributors to that effort may not have sufficient variety of flows to
> evaluate when the migration tools are "done" for the majority of the
> community to have success upgrading to 2.x. A milestone release would allow
> us get more feedback on migration over a longer period than the vote window
> of an RC candidate.
>
> Perhaps we could continue to release from the 1.x line (including minor
> releases with some features) until we are ready to drop the "milestone"
> qualifier from 2.0.0, and only then put 1.x into maintenance-only status.
> It would be the same proposal to move main to target 2.0.0-M1, but relax
> restrictions for what can land on the 1.x branch and be open to a 1.21,
> 1.22, etc. if 2.0.0 work takes longer than anticipated. For example, maybe
> we would be open to landing new/backported processors on the 1.x branch,
> but not core framework features or API changes.
>
> This might not be necessary, but I think it is fair that saying "no new
> features on 1.x" and also "no new features in 2.0.0" puts the project in a
> rough place if 2.0.0 takes longer than a few months, so if we go that
> route, we need to commit to a quick release of 2.0.0 that most users can
> move to easily.
>
> Thanks,
> Kevin
>
> On Jan 11, 2023 at 12:32:46, Joe Witt <jo...@gmail.com> wrote:
>
> > Kevin,
> >
> > Yeah we can do whatever we want as far as 'releases' of 2.0 that are
> prior
> > to us officially considering it 2.0/stable.
> >
> > That said - the migration tooling will be key to provide as we need to
> make
> > the bridge as solid and stable as possible to help someone move from 1.x
> to
> > 2.x. I dont know how related these two concepts (milestone releases and
> > 1.x to 2.x ease really are).
> >
> > Thanks
> >
> > On Wed, Jan 11, 2023 at 10:27 AM Kevin Doran <kd...@apache.org> wrote:
> >
> > [hit the wrong keyboard shortcut, here is the rest of my thoughts]
> >
> >
> > On this point from David:
> >
> >
> > We may need to have a longer release candidate period, or more
> incremental
> >
> > > fix releases
> >
> > > for the initial 2.0.0 release train, but I do not expect delaying a
> 2.0.0
> >
> > > release for new features, as that is not part of the release goals.
> >
> > >
> >
> >
> > Would the community benefit from one or more milestone releases of 2.0,
> to
> >
> > allow for a wider group to run / live on the proposed 2.0 prior to
> >
> > releasing it as "stable"? I know we've never done a milestone release in
> >
> > the past, and I'm not sure what ASF guidance is on the topic, but if it
> >
> > could be beneficial we could look into that.
> >
> >
> > Cheers,
> >
> > Kevin
> >
> >
> > On Jan 11, 2023 at 12:22:43, Kevin Doran <kd...@apache.org> wrote:
> >
> >
> > > I think this is a good, practical discussion.
> >
> > >
> >
> > > On the one hand, we can't put off 2.x any longer as we really need to
> >
> > > updated the minimum required Java to 11. Updating main development to
> >
> > > target 2.x feels like a good way drive progress on that along with the
> >
> > > other 2.0 goals.
> >
> > >
> >
> > > On the other hand, the concerns are valid: moving all development to
> >
> > > target 2.x puts the project at risk if we cannot release 2.0.0 on a
> >
> > > reasonable timeline. The restricted scope of 2.0 helps, but this stated
> >
> > > release goal feels risky to me:
> >
> > >
> >
> > > Implement Migration Tools for Upgrading Flows
> >
> > >
> >
> > >
> >
> > > - Implement automated migration where possible to remap properties
> and
> >
> > > features
> >
> > > - Implement migration tools for manual conversion of XML
> Templates
> >
> > > to JSON Flow Definitions
> >
> > > - Create documentation for manual steps necessary where
> >
> > > programmatic migration cannot be implemented
> >
> > > - NiFi 2.0 should be capable of starting with ghosted components
> >
> > > for removed Processors or Controller Services
> >
> > >
> >
> > >
> >
> > > Removing deprecated components should be fairly straightforward and
> >
> > quick,
> >
> > > but automating and documenting migration is a large effort.
> >
> > >
> >
> > > On this po
> >
> > >
> >
> > >
> >
> > > On Jan 10, 2023 at 09:32:31, Bryan Bende <bb...@gmail.com> wrote:
> >
> > >
> >
> > >> The plan as I understand it is not to diverge and create separate
> >
> > feature
> >
> > >> development on the 1.x line, so I would expect all PRs to continue to
> be
> >
> > >> submitted only to main. We would release 1.x as needed with major bug
> >
> > >> fixes
> >
> > >> or critical security updates, and these would be cherry-picked and/or
> >
> > >> backported as necessary, mostly without the need for PRs, the same as
> we
> >
> > >> would do if we were bringing fixes from main (1.20.0-SNAPSHOT) back
> to a
> >
> > >> maintenance line like (1.19.x). For precedent, we followed this same
> >
> > >> approach going from the 0.x line to 1.0.0 and there wasn't any major
> >
> > >> issue.
> >
> > >>
> >
> > >> On Tue, Jan 10, 2023 at 7:07 AM Otto Fowler <ot...@gmail.com>
> >
> > >> wrote:
> >
> > >>
> >
> > >> It was also mentioned in another thread that we need to have
> agreement
> >
> > on
> >
> > >>
> >
> > >> our explicit strategy and support for 1.x going forward, did that
> >
> > happen?
> >
> > >>
> >
> > >>
> >
> > >> From: Otto Fowler <ot...@gmail.com> <ot...@gmail.com>
> >
> > >>
> >
> > >> Reply: Otto Fowler <ot...@gmail.com> <ottobackwards@gmail.com
> >
> >
> > >>
> >
> > >> Date: January 10, 2023 at 07:02:34
> >
> > >>
> >
> > >> To: dev@nifi.apache.org <de...@nifi.apache.org> <de...@nifi.apache.org>
> >
> > >>
> >
> > >> Subject: Re: [discuss] NiFi 1.20 and NiFi 2.0
> >
> > >>
> >
> > >>
> >
> > >> There needs to be an update to the contributing guide as to how to
> >
> > submit
> >
> > >>
> >
> > >> PRs to 1.x or 2.x etc.
> >
> > >>
> >
> > >>
> >
> > >> From: Joe Witt <jo...@apache.org> <jo...@apache.org>
> >
> > >>
> >
> > >> Reply: dev@nifi.apache.org <de...@nifi.apache.org> <dev@nifi.apache.org
> >
> >
> > >>
> >
> > >> Date: January 9, 2023 at 15:53:16
> >
> > >>
> >
> > >> To: dev@nifi.apache.org <de...@nifi.apache.org> <de...@nifi.apache.org>
> >
> > >>
> >
> > >> Subject: [discuss] NiFi 1.20 and NiFi 2.0
> >
> > >>
> >
> > >>
> >
> > >> Team,
> >
> > >>
> >
> > >>
> >
> > >> As David mentioned in [1] following a successful NiFi 2.0 release goal
> >
> > >>
> >
> > >> planning - we are now going to start moving the 'main' line to be the
> >
> > NiFi
> >
> > >>
> >
> > >> 2.0 line which will allow for the key work to take place. We will also
> >
> > >>
> >
> > >> move niFi 1.x to its appropriate support line.
> >
> > >>
> >
> > >>
> >
> > >> It is also the case that we have nearly 100 JIRAs on NiFi 1.20 and we
> >
> > have
> >
> > >>
> >
> > >> work in there including security items so it is time to make a
> release.
> >
> > >>
> >
> > >> The intent then is to initiate 1.20 and immediate after that change
> >
> > 'main'
> >
> > >>
> >
> > >> to 2.0.
> >
> > >>
> >
> > >>
> >
> > >> Going forward then all work on the 1.x line should be focused on
> >
> > >>
> >
> > >> maintaining existing features, dependencies, and helping 1.x users
> >
> > migrate
> >
> > >>
> >
> > >> to the 2.x line. Otherwise, new feature work will happen on 'main' as
> it
> >
> > >>
> >
> > >> normally does and will come out in the NiFi 2.x release line.
> >
> > >>
> >
> > >>
> >
> > >> Please flag key outstanding items as we narrow down the release
> >
> > candidate
> >
> > >>
> >
> > >> for NiFi 1.20.
> >
> > >>
> >
> > >>
> >
> > >> Thanks
> >
> > >>
> >
> > >> Joe
> >
> > >>
> >
> > >>
> >
> > >> [1] https://lists.apache.org/thread/qo4vvdw46235y7vy2crcd6l4m11wl7jz
> >
> > >>
> >
> > >>
> >
> > >>
> >
> >
> >
>
Re: [discuss] NiFi 1.20 and NiFi 2.0
Posted by Kevin Doran <kd...@apache.org>.
Sorry, Joe, I was not clear, and to be honest the two thoughts are somewhat
unrelated in my mind too :)
I agree that good migration tooling is key. Otherwise, we risk users
staying on 1.x or creating a schism of 1.x and 2.x users.
Good migration tooling will take a while to develop and test, and the core
contributors to that effort may not have sufficient variety of flows to
evaluate when the migration tools are "done" for the majority of the
community to have success upgrading to 2.x. A milestone release would allow
us get more feedback on migration over a longer period than the vote window
of an RC candidate.
Perhaps we could continue to release from the 1.x line (including minor
releases with some features) until we are ready to drop the "milestone"
qualifier from 2.0.0, and only then put 1.x into maintenance-only status.
It would be the same proposal to move main to target 2.0.0-M1, but relax
restrictions for what can land on the 1.x branch and be open to a 1.21,
1.22, etc. if 2.0.0 work takes longer than anticipated. For example, maybe
we would be open to landing new/backported processors on the 1.x branch,
but not core framework features or API changes.
This might not be necessary, but I think it is fair that saying "no new
features on 1.x" and also "no new features in 2.0.0" puts the project in a
rough place if 2.0.0 takes longer than a few months, so if we go that
route, we need to commit to a quick release of 2.0.0 that most users can
move to easily.
Thanks,
Kevin
On Jan 11, 2023 at 12:32:46, Joe Witt <jo...@gmail.com> wrote:
> Kevin,
>
> Yeah we can do whatever we want as far as 'releases' of 2.0 that are prior
> to us officially considering it 2.0/stable.
>
> That said - the migration tooling will be key to provide as we need to make
> the bridge as solid and stable as possible to help someone move from 1.x to
> 2.x. I dont know how related these two concepts (milestone releases and
> 1.x to 2.x ease really are).
>
> Thanks
>
> On Wed, Jan 11, 2023 at 10:27 AM Kevin Doran <kd...@apache.org> wrote:
>
> [hit the wrong keyboard shortcut, here is the rest of my thoughts]
>
>
> On this point from David:
>
>
> We may need to have a longer release candidate period, or more incremental
>
> > fix releases
>
> > for the initial 2.0.0 release train, but I do not expect delaying a 2.0.0
>
> > release for new features, as that is not part of the release goals.
>
> >
>
>
> Would the community benefit from one or more milestone releases of 2.0, to
>
> allow for a wider group to run / live on the proposed 2.0 prior to
>
> releasing it as "stable"? I know we've never done a milestone release in
>
> the past, and I'm not sure what ASF guidance is on the topic, but if it
>
> could be beneficial we could look into that.
>
>
> Cheers,
>
> Kevin
>
>
> On Jan 11, 2023 at 12:22:43, Kevin Doran <kd...@apache.org> wrote:
>
>
> > I think this is a good, practical discussion.
>
> >
>
> > On the one hand, we can't put off 2.x any longer as we really need to
>
> > updated the minimum required Java to 11. Updating main development to
>
> > target 2.x feels like a good way drive progress on that along with the
>
> > other 2.0 goals.
>
> >
>
> > On the other hand, the concerns are valid: moving all development to
>
> > target 2.x puts the project at risk if we cannot release 2.0.0 on a
>
> > reasonable timeline. The restricted scope of 2.0 helps, but this stated
>
> > release goal feels risky to me:
>
> >
>
> > Implement Migration Tools for Upgrading Flows
>
> >
>
> >
>
> > - Implement automated migration where possible to remap properties and
>
> > features
>
> > - Implement migration tools for manual conversion of XML Templates
>
> > to JSON Flow Definitions
>
> > - Create documentation for manual steps necessary where
>
> > programmatic migration cannot be implemented
>
> > - NiFi 2.0 should be capable of starting with ghosted components
>
> > for removed Processors or Controller Services
>
> >
>
> >
>
> > Removing deprecated components should be fairly straightforward and
>
> quick,
>
> > but automating and documenting migration is a large effort.
>
> >
>
> > On this po
>
> >
>
> >
>
> > On Jan 10, 2023 at 09:32:31, Bryan Bende <bb...@gmail.com> wrote:
>
> >
>
> >> The plan as I understand it is not to diverge and create separate
>
> feature
>
> >> development on the 1.x line, so I would expect all PRs to continue to be
>
> >> submitted only to main. We would release 1.x as needed with major bug
>
> >> fixes
>
> >> or critical security updates, and these would be cherry-picked and/or
>
> >> backported as necessary, mostly without the need for PRs, the same as we
>
> >> would do if we were bringing fixes from main (1.20.0-SNAPSHOT) back to a
>
> >> maintenance line like (1.19.x). For precedent, we followed this same
>
> >> approach going from the 0.x line to 1.0.0 and there wasn't any major
>
> >> issue.
>
> >>
>
> >> On Tue, Jan 10, 2023 at 7:07 AM Otto Fowler <ot...@gmail.com>
>
> >> wrote:
>
> >>
>
> >> It was also mentioned in another thread that we need to have agreement
>
> on
>
> >>
>
> >> our explicit strategy and support for 1.x going forward, did that
>
> happen?
>
> >>
>
> >>
>
> >> From: Otto Fowler <ot...@gmail.com> <ot...@gmail.com>
>
> >>
>
> >> Reply: Otto Fowler <ot...@gmail.com> <ot...@gmail.com>
>
> >>
>
> >> Date: January 10, 2023 at 07:02:34
>
> >>
>
> >> To: dev@nifi.apache.org <de...@nifi.apache.org> <de...@nifi.apache.org>
>
> >>
>
> >> Subject: Re: [discuss] NiFi 1.20 and NiFi 2.0
>
> >>
>
> >>
>
> >> There needs to be an update to the contributing guide as to how to
>
> submit
>
> >>
>
> >> PRs to 1.x or 2.x etc.
>
> >>
>
> >>
>
> >> From: Joe Witt <jo...@apache.org> <jo...@apache.org>
>
> >>
>
> >> Reply: dev@nifi.apache.org <de...@nifi.apache.org> <de...@nifi.apache.org>
>
> >>
>
> >> Date: January 9, 2023 at 15:53:16
>
> >>
>
> >> To: dev@nifi.apache.org <de...@nifi.apache.org> <de...@nifi.apache.org>
>
> >>
>
> >> Subject: [discuss] NiFi 1.20 and NiFi 2.0
>
> >>
>
> >>
>
> >> Team,
>
> >>
>
> >>
>
> >> As David mentioned in [1] following a successful NiFi 2.0 release goal
>
> >>
>
> >> planning - we are now going to start moving the 'main' line to be the
>
> NiFi
>
> >>
>
> >> 2.0 line which will allow for the key work to take place. We will also
>
> >>
>
> >> move niFi 1.x to its appropriate support line.
>
> >>
>
> >>
>
> >> It is also the case that we have nearly 100 JIRAs on NiFi 1.20 and we
>
> have
>
> >>
>
> >> work in there including security items so it is time to make a release.
>
> >>
>
> >> The intent then is to initiate 1.20 and immediate after that change
>
> 'main'
>
> >>
>
> >> to 2.0.
>
> >>
>
> >>
>
> >> Going forward then all work on the 1.x line should be focused on
>
> >>
>
> >> maintaining existing features, dependencies, and helping 1.x users
>
> migrate
>
> >>
>
> >> to the 2.x line. Otherwise, new feature work will happen on 'main' as it
>
> >>
>
> >> normally does and will come out in the NiFi 2.x release line.
>
> >>
>
> >>
>
> >> Please flag key outstanding items as we narrow down the release
>
> candidate
>
> >>
>
> >> for NiFi 1.20.
>
> >>
>
> >>
>
> >> Thanks
>
> >>
>
> >> Joe
>
> >>
>
> >>
>
> >> [1] https://lists.apache.org/thread/qo4vvdw46235y7vy2crcd6l4m11wl7jz
>
> >>
>
> >>
>
> >>
>
>
>
Re: [discuss] NiFi 1.20 and NiFi 2.0
Posted by Joe Witt <jo...@gmail.com>.
Kevin,
Yeah we can do whatever we want as far as 'releases' of 2.0 that are prior
to us officially considering it 2.0/stable.
That said - the migration tooling will be key to provide as we need to make
the bridge as solid and stable as possible to help someone move from 1.x to
2.x. I dont know how related these two concepts (milestone releases and
1.x to 2.x ease really are).
Thanks
On Wed, Jan 11, 2023 at 10:27 AM Kevin Doran <kd...@apache.org> wrote:
> [hit the wrong keyboard shortcut, here is the rest of my thoughts]
>
> On this point from David:
>
> We may need to have a longer release candidate period, or more incremental
> > fix releases
> > for the initial 2.0.0 release train, but I do not expect delaying a 2.0.0
> > release for new features, as that is not part of the release goals.
> >
>
> Would the community benefit from one or more milestone releases of 2.0, to
> allow for a wider group to run / live on the proposed 2.0 prior to
> releasing it as "stable"? I know we've never done a milestone release in
> the past, and I'm not sure what ASF guidance is on the topic, but if it
> could be beneficial we could look into that.
>
> Cheers,
> Kevin
>
> On Jan 11, 2023 at 12:22:43, Kevin Doran <kd...@apache.org> wrote:
>
> > I think this is a good, practical discussion.
> >
> > On the one hand, we can't put off 2.x any longer as we really need to
> > updated the minimum required Java to 11. Updating main development to
> > target 2.x feels like a good way drive progress on that along with the
> > other 2.0 goals.
> >
> > On the other hand, the concerns are valid: moving all development to
> > target 2.x puts the project at risk if we cannot release 2.0.0 on a
> > reasonable timeline. The restricted scope of 2.0 helps, but this stated
> > release goal feels risky to me:
> >
> > Implement Migration Tools for Upgrading Flows
> >
> >
> > - Implement automated migration where possible to remap properties and
> > features
> > - Implement migration tools for manual conversion of XML Templates
> > to JSON Flow Definitions
> > - Create documentation for manual steps necessary where
> > programmatic migration cannot be implemented
> > - NiFi 2.0 should be capable of starting with ghosted components
> > for removed Processors or Controller Services
> >
> >
> > Removing deprecated components should be fairly straightforward and
> quick,
> > but automating and documenting migration is a large effort.
> >
> > On this po
> >
> >
> > On Jan 10, 2023 at 09:32:31, Bryan Bende <bb...@gmail.com> wrote:
> >
> >> The plan as I understand it is not to diverge and create separate
> feature
> >> development on the 1.x line, so I would expect all PRs to continue to be
> >> submitted only to main. We would release 1.x as needed with major bug
> >> fixes
> >> or critical security updates, and these would be cherry-picked and/or
> >> backported as necessary, mostly without the need for PRs, the same as we
> >> would do if we were bringing fixes from main (1.20.0-SNAPSHOT) back to a
> >> maintenance line like (1.19.x). For precedent, we followed this same
> >> approach going from the 0.x line to 1.0.0 and there wasn't any major
> >> issue.
> >>
> >> On Tue, Jan 10, 2023 at 7:07 AM Otto Fowler <ot...@gmail.com>
> >> wrote:
> >>
> >> It was also mentioned in another thread that we need to have agreement
> on
> >>
> >> our explicit strategy and support for 1.x going forward, did that
> happen?
> >>
> >>
> >> From: Otto Fowler <ot...@gmail.com> <ot...@gmail.com>
> >>
> >> Reply: Otto Fowler <ot...@gmail.com> <ot...@gmail.com>
> >>
> >> Date: January 10, 2023 at 07:02:34
> >>
> >> To: dev@nifi.apache.org <de...@nifi.apache.org> <de...@nifi.apache.org>
> >>
> >> Subject: Re: [discuss] NiFi 1.20 and NiFi 2.0
> >>
> >>
> >> There needs to be an update to the contributing guide as to how to
> submit
> >>
> >> PRs to 1.x or 2.x etc.
> >>
> >>
> >> From: Joe Witt <jo...@apache.org> <jo...@apache.org>
> >>
> >> Reply: dev@nifi.apache.org <de...@nifi.apache.org> <de...@nifi.apache.org>
> >>
> >> Date: January 9, 2023 at 15:53:16
> >>
> >> To: dev@nifi.apache.org <de...@nifi.apache.org> <de...@nifi.apache.org>
> >>
> >> Subject: [discuss] NiFi 1.20 and NiFi 2.0
> >>
> >>
> >> Team,
> >>
> >>
> >> As David mentioned in [1] following a successful NiFi 2.0 release goal
> >>
> >> planning - we are now going to start moving the 'main' line to be the
> NiFi
> >>
> >> 2.0 line which will allow for the key work to take place. We will also
> >>
> >> move niFi 1.x to its appropriate support line.
> >>
> >>
> >> It is also the case that we have nearly 100 JIRAs on NiFi 1.20 and we
> have
> >>
> >> work in there including security items so it is time to make a release.
> >>
> >> The intent then is to initiate 1.20 and immediate after that change
> 'main'
> >>
> >> to 2.0.
> >>
> >>
> >> Going forward then all work on the 1.x line should be focused on
> >>
> >> maintaining existing features, dependencies, and helping 1.x users
> migrate
> >>
> >> to the 2.x line. Otherwise, new feature work will happen on 'main' as it
> >>
> >> normally does and will come out in the NiFi 2.x release line.
> >>
> >>
> >> Please flag key outstanding items as we narrow down the release
> candidate
> >>
> >> for NiFi 1.20.
> >>
> >>
> >> Thanks
> >>
> >> Joe
> >>
> >>
> >> [1] https://lists.apache.org/thread/qo4vvdw46235y7vy2crcd6l4m11wl7jz
> >>
> >>
> >>
>
Re: [discuss] NiFi 1.20 and NiFi 2.0
Posted by Kevin Doran <kd...@apache.org>.
[hit the wrong keyboard shortcut, here is the rest of my thoughts]
On this point from David:
We may need to have a longer release candidate period, or more incremental
> fix releases
> for the initial 2.0.0 release train, but I do not expect delaying a 2.0.0
> release for new features, as that is not part of the release goals.
>
Would the community benefit from one or more milestone releases of 2.0, to
allow for a wider group to run / live on the proposed 2.0 prior to
releasing it as "stable"? I know we've never done a milestone release in
the past, and I'm not sure what ASF guidance is on the topic, but if it
could be beneficial we could look into that.
Cheers,
Kevin
On Jan 11, 2023 at 12:22:43, Kevin Doran <kd...@apache.org> wrote:
> I think this is a good, practical discussion.
>
> On the one hand, we can't put off 2.x any longer as we really need to
> updated the minimum required Java to 11. Updating main development to
> target 2.x feels like a good way drive progress on that along with the
> other 2.0 goals.
>
> On the other hand, the concerns are valid: moving all development to
> target 2.x puts the project at risk if we cannot release 2.0.0 on a
> reasonable timeline. The restricted scope of 2.0 helps, but this stated
> release goal feels risky to me:
>
> Implement Migration Tools for Upgrading Flows
>
>
> - Implement automated migration where possible to remap properties and
> features
> - Implement migration tools for manual conversion of XML Templates
> to JSON Flow Definitions
> - Create documentation for manual steps necessary where
> programmatic migration cannot be implemented
> - NiFi 2.0 should be capable of starting with ghosted components
> for removed Processors or Controller Services
>
>
> Removing deprecated components should be fairly straightforward and quick,
> but automating and documenting migration is a large effort.
>
> On this po
>
>
> On Jan 10, 2023 at 09:32:31, Bryan Bende <bb...@gmail.com> wrote:
>
>> The plan as I understand it is not to diverge and create separate feature
>> development on the 1.x line, so I would expect all PRs to continue to be
>> submitted only to main. We would release 1.x as needed with major bug
>> fixes
>> or critical security updates, and these would be cherry-picked and/or
>> backported as necessary, mostly without the need for PRs, the same as we
>> would do if we were bringing fixes from main (1.20.0-SNAPSHOT) back to a
>> maintenance line like (1.19.x). For precedent, we followed this same
>> approach going from the 0.x line to 1.0.0 and there wasn't any major
>> issue.
>>
>> On Tue, Jan 10, 2023 at 7:07 AM Otto Fowler <ot...@gmail.com>
>> wrote:
>>
>> It was also mentioned in another thread that we need to have agreement on
>>
>> our explicit strategy and support for 1.x going forward, did that happen?
>>
>>
>> From: Otto Fowler <ot...@gmail.com> <ot...@gmail.com>
>>
>> Reply: Otto Fowler <ot...@gmail.com> <ot...@gmail.com>
>>
>> Date: January 10, 2023 at 07:02:34
>>
>> To: dev@nifi.apache.org <de...@nifi.apache.org> <de...@nifi.apache.org>
>>
>> Subject: Re: [discuss] NiFi 1.20 and NiFi 2.0
>>
>>
>> There needs to be an update to the contributing guide as to how to submit
>>
>> PRs to 1.x or 2.x etc.
>>
>>
>> From: Joe Witt <jo...@apache.org> <jo...@apache.org>
>>
>> Reply: dev@nifi.apache.org <de...@nifi.apache.org> <de...@nifi.apache.org>
>>
>> Date: January 9, 2023 at 15:53:16
>>
>> To: dev@nifi.apache.org <de...@nifi.apache.org> <de...@nifi.apache.org>
>>
>> Subject: [discuss] NiFi 1.20 and NiFi 2.0
>>
>>
>> Team,
>>
>>
>> As David mentioned in [1] following a successful NiFi 2.0 release goal
>>
>> planning - we are now going to start moving the 'main' line to be the NiFi
>>
>> 2.0 line which will allow for the key work to take place. We will also
>>
>> move niFi 1.x to its appropriate support line.
>>
>>
>> It is also the case that we have nearly 100 JIRAs on NiFi 1.20 and we have
>>
>> work in there including security items so it is time to make a release.
>>
>> The intent then is to initiate 1.20 and immediate after that change 'main'
>>
>> to 2.0.
>>
>>
>> Going forward then all work on the 1.x line should be focused on
>>
>> maintaining existing features, dependencies, and helping 1.x users migrate
>>
>> to the 2.x line. Otherwise, new feature work will happen on 'main' as it
>>
>> normally does and will come out in the NiFi 2.x release line.
>>
>>
>> Please flag key outstanding items as we narrow down the release candidate
>>
>> for NiFi 1.20.
>>
>>
>> Thanks
>>
>> Joe
>>
>>
>> [1] https://lists.apache.org/thread/qo4vvdw46235y7vy2crcd6l4m11wl7jz
>>
>>
>>
Re: [discuss] NiFi 1.20 and NiFi 2.0
Posted by Kevin Doran <kd...@apache.org>.
I think this is a good, practical discussion.
On the one hand, we can't put off 2.x any longer as we really need to
updated the minimum required Java to 11. Updating main development to
target 2.x feels like a good way drive progress on that along with the
other 2.0 goals.
On the other hand, the concerns are valid: moving all development to target
2.x puts the project at risk if we cannot release 2.0.0 on a reasonable
timeline. The restricted scope of 2.0 helps, but this stated release goal
feels risky to me:
Implement Migration Tools for Upgrading Flows
- Implement automated migration where possible to remap properties and
features
- Implement migration tools for manual conversion of XML Templates to
JSON Flow Definitions
- Create documentation for manual steps necessary where programmatic
migration cannot be implemented
- NiFi 2.0 should be capable of starting with ghosted components for
removed Processors or Controller Services
Removing deprecated components should be fairly straightforward and quick,
but automating and documenting migration is a large effort.
On this po
On Jan 10, 2023 at 09:32:31, Bryan Bende <bb...@gmail.com> wrote:
> The plan as I understand it is not to diverge and create separate feature
> development on the 1.x line, so I would expect all PRs to continue to be
> submitted only to main. We would release 1.x as needed with major bug fixes
> or critical security updates, and these would be cherry-picked and/or
> backported as necessary, mostly without the need for PRs, the same as we
> would do if we were bringing fixes from main (1.20.0-SNAPSHOT) back to a
> maintenance line like (1.19.x). For precedent, we followed this same
> approach going from the 0.x line to 1.0.0 and there wasn't any major issue.
>
> On Tue, Jan 10, 2023 at 7:07 AM Otto Fowler <ot...@gmail.com>
> wrote:
>
> It was also mentioned in another thread that we need to have agreement on
>
> our explicit strategy and support for 1.x going forward, did that happen?
>
>
> From: Otto Fowler <ot...@gmail.com> <ot...@gmail.com>
>
> Reply: Otto Fowler <ot...@gmail.com> <ot...@gmail.com>
>
> Date: January 10, 2023 at 07:02:34
>
> To: dev@nifi.apache.org <de...@nifi.apache.org> <de...@nifi.apache.org>
>
> Subject: Re: [discuss] NiFi 1.20 and NiFi 2.0
>
>
> There needs to be an update to the contributing guide as to how to submit
>
> PRs to 1.x or 2.x etc.
>
>
> From: Joe Witt <jo...@apache.org> <jo...@apache.org>
>
> Reply: dev@nifi.apache.org <de...@nifi.apache.org> <de...@nifi.apache.org>
>
> Date: January 9, 2023 at 15:53:16
>
> To: dev@nifi.apache.org <de...@nifi.apache.org> <de...@nifi.apache.org>
>
> Subject: [discuss] NiFi 1.20 and NiFi 2.0
>
>
> Team,
>
>
> As David mentioned in [1] following a successful NiFi 2.0 release goal
>
> planning - we are now going to start moving the 'main' line to be the NiFi
>
> 2.0 line which will allow for the key work to take place. We will also
>
> move niFi 1.x to its appropriate support line.
>
>
> It is also the case that we have nearly 100 JIRAs on NiFi 1.20 and we have
>
> work in there including security items so it is time to make a release.
>
> The intent then is to initiate 1.20 and immediate after that change 'main'
>
> to 2.0.
>
>
> Going forward then all work on the 1.x line should be focused on
>
> maintaining existing features, dependencies, and helping 1.x users migrate
>
> to the 2.x line. Otherwise, new feature work will happen on 'main' as it
>
> normally does and will come out in the NiFi 2.x release line.
>
>
> Please flag key outstanding items as we narrow down the release candidate
>
> for NiFi 1.20.
>
>
> Thanks
>
> Joe
>
>
> [1] https://lists.apache.org/thread/qo4vvdw46235y7vy2crcd6l4m11wl7jz
>
>
>
Re: [discuss] NiFi 1.20 and NiFi 2.0
Posted by Bryan Bende <bb...@gmail.com>.
The plan as I understand it is not to diverge and create separate feature
development on the 1.x line, so I would expect all PRs to continue to be
submitted only to main. We would release 1.x as needed with major bug fixes
or critical security updates, and these would be cherry-picked and/or
backported as necessary, mostly without the need for PRs, the same as we
would do if we were bringing fixes from main (1.20.0-SNAPSHOT) back to a
maintenance line like (1.19.x). For precedent, we followed this same
approach going from the 0.x line to 1.0.0 and there wasn't any major issue.
On Tue, Jan 10, 2023 at 7:07 AM Otto Fowler <ot...@gmail.com> wrote:
> It was also mentioned in another thread that we need to have agreement on
> our explicit strategy and support for 1.x going forward, did that happen?
>
> From: Otto Fowler <ot...@gmail.com> <ot...@gmail.com>
> Reply: Otto Fowler <ot...@gmail.com> <ot...@gmail.com>
> Date: January 10, 2023 at 07:02:34
> To: dev@nifi.apache.org <de...@nifi.apache.org> <de...@nifi.apache.org>
> Subject: Re: [discuss] NiFi 1.20 and NiFi 2.0
>
> There needs to be an update to the contributing guide as to how to submit
> PRs to 1.x or 2.x etc.
>
> From: Joe Witt <jo...@apache.org> <jo...@apache.org>
> Reply: dev@nifi.apache.org <de...@nifi.apache.org> <de...@nifi.apache.org>
> Date: January 9, 2023 at 15:53:16
> To: dev@nifi.apache.org <de...@nifi.apache.org> <de...@nifi.apache.org>
> Subject: [discuss] NiFi 1.20 and NiFi 2.0
>
> Team,
>
> As David mentioned in [1] following a successful NiFi 2.0 release goal
> planning - we are now going to start moving the 'main' line to be the NiFi
> 2.0 line which will allow for the key work to take place. We will also
> move niFi 1.x to its appropriate support line.
>
> It is also the case that we have nearly 100 JIRAs on NiFi 1.20 and we have
> work in there including security items so it is time to make a release.
> The intent then is to initiate 1.20 and immediate after that change 'main'
> to 2.0.
>
> Going forward then all work on the 1.x line should be focused on
> maintaining existing features, dependencies, and helping 1.x users migrate
> to the 2.x line. Otherwise, new feature work will happen on 'main' as it
> normally does and will come out in the NiFi 2.x release line.
>
> Please flag key outstanding items as we narrow down the release candidate
> for NiFi 1.20.
>
> Thanks
> Joe
>
> [1] https://lists.apache.org/thread/qo4vvdw46235y7vy2crcd6l4m11wl7jz
>
Re: [discuss] NiFi 1.20 and NiFi 2.0
Posted by Otto Fowler <ot...@gmail.com>.
It was also mentioned in another thread that we need to have agreement on
our explicit strategy and support for 1.x going forward, did that happen?
From: Otto Fowler <ot...@gmail.com> <ot...@gmail.com>
Reply: Otto Fowler <ot...@gmail.com> <ot...@gmail.com>
Date: January 10, 2023 at 07:02:34
To: dev@nifi.apache.org <de...@nifi.apache.org> <de...@nifi.apache.org>
Subject: Re: [discuss] NiFi 1.20 and NiFi 2.0
There needs to be an update to the contributing guide as to how to submit
PRs to 1.x or 2.x etc.
From: Joe Witt <jo...@apache.org> <jo...@apache.org>
Reply: dev@nifi.apache.org <de...@nifi.apache.org> <de...@nifi.apache.org>
Date: January 9, 2023 at 15:53:16
To: dev@nifi.apache.org <de...@nifi.apache.org> <de...@nifi.apache.org>
Subject: [discuss] NiFi 1.20 and NiFi 2.0
Team,
As David mentioned in [1] following a successful NiFi 2.0 release goal
planning - we are now going to start moving the 'main' line to be the NiFi
2.0 line which will allow for the key work to take place. We will also
move niFi 1.x to its appropriate support line.
It is also the case that we have nearly 100 JIRAs on NiFi 1.20 and we have
work in there including security items so it is time to make a release.
The intent then is to initiate 1.20 and immediate after that change 'main'
to 2.0.
Going forward then all work on the 1.x line should be focused on
maintaining existing features, dependencies, and helping 1.x users migrate
to the 2.x line. Otherwise, new feature work will happen on 'main' as it
normally does and will come out in the NiFi 2.x release line.
Please flag key outstanding items as we narrow down the release candidate
for NiFi 1.20.
Thanks
Joe
[1] https://lists.apache.org/thread/qo4vvdw46235y7vy2crcd6l4m11wl7jz
Re: [discuss] NiFi 1.20 and NiFi 2.0
Posted by Otto Fowler <ot...@gmail.com>.
There needs to be an update to the contributing guide as to how to submit
PRs to 1.x or 2.x etc.
From: Joe Witt <jo...@apache.org> <jo...@apache.org>
Reply: dev@nifi.apache.org <de...@nifi.apache.org> <de...@nifi.apache.org>
Date: January 9, 2023 at 15:53:16
To: dev@nifi.apache.org <de...@nifi.apache.org> <de...@nifi.apache.org>
Subject: [discuss] NiFi 1.20 and NiFi 2.0
Team,
As David mentioned in [1] following a successful NiFi 2.0 release goal
planning - we are now going to start moving the 'main' line to be the NiFi
2.0 line which will allow for the key work to take place. We will also
move niFi 1.x to its appropriate support line.
It is also the case that we have nearly 100 JIRAs on NiFi 1.20 and we have
work in there including security items so it is time to make a release.
The intent then is to initiate 1.20 and immediate after that change 'main'
to 2.0.
Going forward then all work on the 1.x line should be focused on
maintaining existing features, dependencies, and helping 1.x users migrate
to the 2.x line. Otherwise, new feature work will happen on 'main' as it
normally does and will come out in the NiFi 2.x release line.
Please flag key outstanding items as we narrow down the release candidate
for NiFi 1.20.
Thanks
Joe
[1] https://lists.apache.org/thread/qo4vvdw46235y7vy2crcd6l4m11wl7jz