Re: [DISCUSS] Status of Statefun Project

[Galen Warren via user <us...@flink.apache.org>]

Hi Martijn, since you opened this discussion thread, I'm curious what your
thoughts are in light of the responses? Thanks.

On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <ga...@cvillewarrens.com>
wrote:

> I use Apache Flink for stream processing, and StateFun as a hand-off point
>> for the rest of the application.
>> It serves well as a bridge between a Flink Streaming job and
>> micro-services.
>
>
> This is essentially how I use it as well, and I would also be sad to see
> it sunsetted. It works well; I don't know that there is a lot of new
> development required, but if there are no new Statefun releases, then
> Statefun can only be used with older Flink versions.
>
> On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
> mvillalobos@kineteque.com> wrote:
>
>> I am currently using Stateful Functions in my application.
>>
>> I use Apache Flink for stream processing, and StateFun as a hand-off
>> point for the rest of the application.
>> It serves well as a bridge between a Flink Streaming job and
>> micro-services.
>>
>> I would be disappointed if StateFun was sunsetted.  Its a good idea.
>>
>> If there is anything I can do to help, as a contributor perhaps, please
>> let me know.
>>
>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <ma...@apache.org>
>> wrote:
>> >
>> > Hi everyone,
>> >
>> > I want to open a discussion on the status of the Statefun Project [1]
>> in Apache Flink. As you might have noticed, there hasn't been much
>> development over the past months in the Statefun repository [2]. There is
>> currently a lack of active contributors and committers who are able to help
>> with the maintenance of the project.
>> >
>> > In order to improve the situation, we need to solve the lack of
>> committers and the lack of contributors.
>> >
>> > On the lack of committers:
>> >
>> > 1. Ideally, there are some of the current Flink committers who have the
>> bandwidth and can help with reviewing PRs and merging them.
>> > 2. If that's not an option, it could be a consideration that current
>> committers only approve and review PRs, that are approved by those who are
>> willing to contribute to Statefun and if the CI passes
>> >
>> > On the lack of contributors:
>> >
>> > 3. Next to having this discussion on the Dev and User mailing list, we
>> can also create a blog with a call for new contributors on the Flink
>> project website, send out some tweets on the Flink / Statefun twitter
>> accounts, post messages on Slack etc. In that message, we would inform how
>> those that are interested in contributing can start and where they could
>> reach out for more information.
>> >
>> > There's also option 4. where a group of interested people would split
>> Statefun from the Flink project and make it a separate top level project
>> under the Apache Flink umbrella (similar as recently has happened with
>> Flink Table Store, which has become Apache Paimon).
>> >
>> > If we see no improvements in the coming period, we should consider
>> sunsetting Statefun and communicate that clearly to the users.
>> >
>> > I'm looking forward to your thoughts.
>> >
>> > Best regards,
>> >
>> > Martijn
>> >
>> > [1] https://nightlies.apache.org/flink/flink-statefun-docs-master/ <
>> https://nightlies.apache.org/flink/flink-statefun-docs-master/>
>> > [2] https://github.com/apache/flink-statefun <
>> https://github.com/apache/flink-statefun>
>>
>

Re: [DISCUSS] Status of Statefun Project

[Galen Warren <ga...@cvillewarrens.com.INVALID>]

Thanks for the information.

On Tue, Jun 6, 2023, 6:07 AM Chesnay Schepler <ch...@apache.org> wrote:

> If you were to fork it /and want to redistribute it/ then the short
> version is that
>
>  1. you have to adhere to the Apache licensing requirements
>  2. you have to make it clear that your fork does not belong to the
>     Apache Flink project. (Trademarks and all that)
>
> Neither should be significant hurdles (there should also be plenty of
> online resources regarding 1), and if you do this then you can freely
> share your fork with others.
>
> I've also pinged Martijn to take a look at this thread.
> To my knowledge the project hasn't decided anything yet.
>
> On 27/05/2023 04:05, Galen Warren wrote:
> > Ok, I get it. No interest.
> >
> > If this project is being abandoned, I guess I'll work with my own fork.
> Is
> > there anything I should consider here? Can I share it with other people
> who
> > use this project?
> >
> > On Tue, May 16, 2023 at 10:50 AM Galen Warren<ga...@cvillewarrens.com>
> > wrote:
> >
> >> Hi Martijn, since you opened this discussion thread, I'm curious what
> your
> >> thoughts are in light of the responses? Thanks.
> >>
> >> On Wed, Apr 19, 2023 at 1:21 PM Galen Warren<ga...@cvillewarrens.com>
> >> wrote:
> >>
> >>> I use Apache Flink for stream processing, and StateFun as a hand-off
> >>>> point for the rest of the application.
> >>>> It serves well as a bridge between a Flink Streaming job and
> >>>> micro-services.
> >>>
> >>> This is essentially how I use it as well, and I would also be sad to
> see
> >>> it sunsetted. It works well; I don't know that there is a lot of new
> >>> development required, but if there are no new Statefun releases, then
> >>> Statefun can only be used with older Flink versions.
> >>>
> >>> On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
> >>> mvillalobos@kineteque.com> wrote:
> >>>
> >>>> I am currently using Stateful Functions in my application.
> >>>>
> >>>> I use Apache Flink for stream processing, and StateFun as a hand-off
> >>>> point for the rest of the application.
> >>>> It serves well as a bridge between a Flink Streaming job and
> >>>> micro-services.
> >>>>
> >>>> I would be disappointed if StateFun was sunsetted.  Its a good idea.
> >>>>
> >>>> If there is anything I can do to help, as a contributor perhaps,
> please
> >>>> let me know.
> >>>>
> >>>>> On Apr 3, 2023, at 2:02 AM, Martijn Visser<ma...@apache.org>
> >>>> wrote:
> >>>>> Hi everyone,
> >>>>>
> >>>>> I want to open a discussion on the status of the Statefun Project [1]
> >>>> in Apache Flink. As you might have noticed, there hasn't been much
> >>>> development over the past months in the Statefun repository [2].
> There is
> >>>> currently a lack of active contributors and committers who are able
> to help
> >>>> with the maintenance of the project.
> >>>>> In order to improve the situation, we need to solve the lack of
> >>>> committers and the lack of contributors.
> >>>>> On the lack of committers:
> >>>>>
> >>>>> 1. Ideally, there are some of the current Flink committers who have
> >>>> the bandwidth and can help with reviewing PRs and merging them.
> >>>>> 2. If that's not an option, it could be a consideration that current
> >>>> committers only approve and review PRs, that are approved by those
> who are
> >>>> willing to contribute to Statefun and if the CI passes
> >>>>> On the lack of contributors:
> >>>>>
> >>>>> 3. Next to having this discussion on the Dev and User mailing list,
> we
> >>>> can also create a blog with a call for new contributors on the Flink
> >>>> project website, send out some tweets on the Flink / Statefun twitter
> >>>> accounts, post messages on Slack etc. In that message, we would
> inform how
> >>>> those that are interested in contributing can start and where they
> could
> >>>> reach out for more information.
> >>>>> There's also option 4. where a group of interested people would split
> >>>> Statefun from the Flink project and make it a separate top level
> project
> >>>> under the Apache Flink umbrella (similar as recently has happened with
> >>>> Flink Table Store, which has become Apache Paimon).
> >>>>> If we see no improvements in the coming period, we should consider
> >>>> sunsetting Statefun and communicate that clearly to the users.
> >>>>> I'm looking forward to your thoughts.
> >>>>>
> >>>>> Best regards,
> >>>>>
> >>>>> Martijn
> >>>>>
> >>>>> [1]https://nightlies.apache.org/flink/flink-statefun-docs-master/  <
> >>>> https://nightlies.apache.org/flink/flink-statefun-docs-master/>
> >>>>> [2]https://github.com/apache/flink-statefun  <
> >>>> https://github.com/apache/flink-statefun>
> >>>>
>

Re: [DISCUSS] Status of Statefun Project

[Filip Karnicki <fi...@gmail.com>]

This is great news, as we're using statefun as well.

Please don't hesitate to let me know if you need me to do some additional
testing on a real life prod-like setup.

On Sat, 24 Jun 2023 at 18:41, Galen Warren via user <us...@flink.apache.org>
wrote:

> Great -- thanks!
>
> I'm going to be out of town for about a week but I'll take a look at this
> when I'm back.
>
> On Tue, Jun 20, 2023 at 8:46 AM Martijn Visser <mv...@confluent.io>
> wrote:
>
>> Hi Galen,
>>
>> Yes, I'll be more than happy to help with Statefun releases.
>>
>> Best regards,
>>
>> Martijn
>>
>> On Tue, Jun 20, 2023 at 2:21 PM Galen Warren <ga...@cvillewarrens.com>
>> wrote:
>>
>>> Thanks.
>>>
>>> Martijn, to answer your question, I'd need to do a small amount of work
>>> to get a PR ready, but not much. Happy to do it if we're deciding to
>>> restart Statefun releases -- are we?
>>>
>>> -- Galen
>>>
>>> On Sat, Jun 17, 2023 at 9:47 AM Tzu-Li (Gordon) Tai <tz...@apache.org>
>>> wrote:
>>>
>>>> > Perhaps he could weigh in on whether the combination of automated
>>>> tests plus those smoke tests should be sufficient for testing with new
>>>> Flink versions
>>>>
>>>> What we usually did at the bare minimum for new StateFun releases was
>>>> the following:
>>>>
>>>>    1. Build tests (including the smoke tests in the e2e module, which
>>>>    covers important tests like exactly-once verification)
>>>>    2. Updating the flink-statefun-playground repo and manually running
>>>>    all language examples there.
>>>>
>>>> If upgrading Flink versions was the only change in the release, I'd
>>>> probably say that this is sufficient.
>>>>
>>>> Best,
>>>> Gordon
>>>>
>>>> On Thu, Jun 15, 2023 at 5:25 AM Martijn Visser <
>>>> martijnvisser@apache.org> wrote:
>>>>
>>>>> Let me know if you have a PR for a Flink update :)
>>>>>
>>>>> On Thu, Jun 8, 2023 at 5:52 PM Galen Warren via user <
>>>>> user@flink.apache.org> wrote:
>>>>>
>>>>>> Thanks Martijn.
>>>>>>
>>>>>> Personally, I'm already using a local fork of Statefun that is
>>>>>> compatible with Flink 1.16.x, so I wouldn't have any need for a released
>>>>>> version compatible with 1.15.x. I'd be happy to do the PRs to modify
>>>>>> Statefun to work with new versions of Flink as they come along.
>>>>>>
>>>>>> As for testing, Statefun does have unit tests and Gordon also sent me
>>>>>> instructions a while back for how to do some additional smoke tests which
>>>>>> are pretty straightforward. Perhaps he could weigh in on whether the
>>>>>> combination of automated tests plus those smoke tests should be sufficient
>>>>>> for testing with new Flink versions (I believe the answer is yes).
>>>>>>
>>>>>> -- Galen
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Jun 8, 2023 at 8:01 AM Martijn Visser <
>>>>>> martijnvisser@apache.org> wrote:
>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> Apologies for the late reply.
>>>>>>>
>>>>>>> I'm willing to help out with merging requests in Statefun to keep
>>>>>>> them
>>>>>>> compatible with new Flink releases and create new releases. I do
>>>>>>> think that
>>>>>>> validation of the functionality of these releases depends a lot on
>>>>>>> those
>>>>>>> who do these compatibility updates, with PMC members helping out
>>>>>>> with the
>>>>>>> formal process.
>>>>>>>
>>>>>>> > Why can't the Apache Software Foundation allow community members
>>>>>>> to bring
>>>>>>> it up to date?
>>>>>>>
>>>>>>> There's nothing preventing anyone from reviewing any of the current
>>>>>>> PRs or
>>>>>>> opening new ones. However, none of them are approved [1], so there's
>>>>>>> also
>>>>>>> nothing to merge.
>>>>>>>
>>>>>>> > I believe that there are people and companies on this mailing list
>>>>>>> interested in supporting Apache Flink Stateful Functions.
>>>>>>>
>>>>>>> If so, then now is the time to show.
>>>>>>>
>>>>>>> Would there be a preference to create a release with Galen's merged
>>>>>>> compatibility update to Flink 1.15.2, or do we want to skip that and
>>>>>>> go
>>>>>>> straight to a newer version?
>>>>>>>
>>>>>>> Best regards,
>>>>>>>
>>>>>>> Martijn
>>>>>>>
>>>>>>> [1]
>>>>>>>
>>>>>>> https://github.com/apache/flink-statefun/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved
>>>>>>>
>>>>>>> On Tue, Jun 6, 2023 at 3:55 PM Marco Villalobos <
>>>>>>> mvillalobos@kineteque.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> > Why can't the Apache Software Foundation allow community members
>>>>>>> to bring
>>>>>>> > it up to date?
>>>>>>> >
>>>>>>> > What's the process for that?
>>>>>>> >
>>>>>>> > I believe that there are people and companies on this mailing list
>>>>>>> > interested in supporting Apache Flink Stateful Functions.
>>>>>>> >
>>>>>>> > You already had two people on this thread express interest.
>>>>>>> >
>>>>>>> > At the very least, we could keep the library versions up to date.
>>>>>>> >
>>>>>>> > There are only a small list of new features that might be
>>>>>>> worthwhile:
>>>>>>> >
>>>>>>> > 1. event time processing
>>>>>>> > 2. state rest api
>>>>>>> >
>>>>>>> >
>>>>>>> > On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <ch...@apache.org>
>>>>>>> wrote:
>>>>>>> >
>>>>>>> > If you were to fork it *and want to redistribute it* then the short
>>>>>>> > version is that
>>>>>>> >
>>>>>>> >    1. you have to adhere to the Apache licensing requirements
>>>>>>> >    2. you have to make it clear that your fork does not belong to
>>>>>>> the
>>>>>>> >    Apache Flink project. (Trademarks and all that)
>>>>>>> >
>>>>>>> > Neither should be significant hurdles (there should also be plenty
>>>>>>> of
>>>>>>> > online resources regarding 1), and if you do this then you can
>>>>>>> freely share
>>>>>>> > your fork with others.
>>>>>>> >
>>>>>>> > I've also pinged Martijn to take a look at this thread.
>>>>>>> > To my knowledge the project hasn't decided anything yet.
>>>>>>> >
>>>>>>> > On 27/05/2023 04:05, Galen Warren wrote:
>>>>>>> >
>>>>>>> > Ok, I get it. No interest.
>>>>>>> >
>>>>>>> > If this project is being abandoned, I guess I'll work with my own
>>>>>>> fork. Is
>>>>>>> > there anything I should consider here? Can I share it with other
>>>>>>> people who
>>>>>>> > use this project?
>>>>>>> >
>>>>>>> > On Tue, May 16, 2023 at 10:50 AM Galen Warren <
>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
>>>>>>> > wrote:
>>>>>>> >
>>>>>>> >
>>>>>>> > Hi Martijn, since you opened this discussion thread, I'm curious
>>>>>>> what your
>>>>>>> > thoughts are in light of the responses? Thanks.
>>>>>>> >
>>>>>>> > On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <
>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
>>>>>>> > wrote:
>>>>>>> >
>>>>>>> >
>>>>>>> > I use Apache Flink for stream processing, and StateFun as a
>>>>>>> hand-off
>>>>>>> >
>>>>>>> > point for the rest of the application.
>>>>>>> > It serves well as a bridge between a Flink Streaming job and
>>>>>>> > micro-services.
>>>>>>> >
>>>>>>> > This is essentially how I use it as well, and I would also be sad
>>>>>>> to see
>>>>>>> > it sunsetted. It works well; I don't know that there is a lot of
>>>>>>> new
>>>>>>> > development required, but if there are no new Statefun releases,
>>>>>>> then
>>>>>>> > Statefun can only be used with older Flink versions.
>>>>>>> >
>>>>>>> > On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
>>>>>>> mvillalobos@kineteque.com> wrote:
>>>>>>> >
>>>>>>> >
>>>>>>> > I am currently using Stateful Functions in my application.
>>>>>>> >
>>>>>>> > I use Apache Flink for stream processing, and StateFun as a
>>>>>>> hand-off
>>>>>>> > point for the rest of the application.
>>>>>>> > It serves well as a bridge between a Flink Streaming job and
>>>>>>> > micro-services.
>>>>>>> >
>>>>>>> > I would be disappointed if StateFun was sunsetted.  Its a good
>>>>>>> idea.
>>>>>>> >
>>>>>>> > If there is anything I can do to help, as a contributor perhaps,
>>>>>>> please
>>>>>>> > let me know.
>>>>>>> >
>>>>>>> >
>>>>>>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <
>>>>>>> martijnvisser@apache.org> <ma...@apache.org>
>>>>>>> >
>>>>>>> > wrote:
>>>>>>> >
>>>>>>> > Hi everyone,
>>>>>>> >
>>>>>>> > I want to open a discussion on the status of the Statefun Project
>>>>>>> [1]
>>>>>>> >
>>>>>>> > in Apache Flink. As you might have noticed, there hasn't been much
>>>>>>> > development over the past months in the Statefun repository [2].
>>>>>>> There is
>>>>>>> > currently a lack of active contributors and committers who are
>>>>>>> able to help
>>>>>>> > with the maintenance of the project.
>>>>>>> >
>>>>>>> > In order to improve the situation, we need to solve the lack of
>>>>>>> >
>>>>>>> > committers and the lack of contributors.
>>>>>>> >
>>>>>>> > On the lack of committers:
>>>>>>> >
>>>>>>> > 1. Ideally, there are some of the current Flink committers who have
>>>>>>> >
>>>>>>> > the bandwidth and can help with reviewing PRs and merging them.
>>>>>>> >
>>>>>>> > 2. If that's not an option, it could be a consideration that
>>>>>>> current
>>>>>>> >
>>>>>>> > committers only approve and review PRs, that are approved by those
>>>>>>> who are
>>>>>>> > willing to contribute to Statefun and if the CI passes
>>>>>>> >
>>>>>>> > On the lack of contributors:
>>>>>>> >
>>>>>>> > 3. Next to having this discussion on the Dev and User mailing
>>>>>>> list, we
>>>>>>> >
>>>>>>> > can also create a blog with a call for new contributors on the
>>>>>>> Flink
>>>>>>> > project website, send out some tweets on the Flink / Statefun
>>>>>>> twitter
>>>>>>> > accounts, post messages on Slack etc. In that message, we would
>>>>>>> inform how
>>>>>>> > those that are interested in contributing can start and where they
>>>>>>> could
>>>>>>> > reach out for more information.
>>>>>>> >
>>>>>>> > There's also option 4. where a group of interested people would
>>>>>>> split
>>>>>>> >
>>>>>>> > Statefun from the Flink project and make it a separate top level
>>>>>>> project
>>>>>>> > under the Apache Flink umbrella (similar as recently has happened
>>>>>>> with
>>>>>>> > Flink Table Store, which has become Apache Paimon).
>>>>>>> >
>>>>>>> > If we see no improvements in the coming period, we should consider
>>>>>>> >
>>>>>>> > sunsetting Statefun and communicate that clearly to the users.
>>>>>>> >
>>>>>>> > I'm looking forward to your thoughts.
>>>>>>> >
>>>>>>> > Best regards,
>>>>>>> >
>>>>>>> > Martijn
>>>>>>> >
>>>>>>> > [1] https://nightlies.apache.org/flink/flink-statefun-docs-master/
>>>>>>> <
>>>>>>> >
>>>>>>> > https://nightlies.apache.org/flink/flink-statefun-docs-master/>
>>>>>>> >
>>>>>>> > [2] https://github.com/apache/flink-statefun <
>>>>>>> >
>>>>>>> > https://github.com/apache/flink-statefun>
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>>
>>>>>>

Re: [DISCUSS] Status of Statefun Project

[Martijn Visser <ma...@apache.org>]

Hi all,

@Gale thanks for sharing that insight.

@Frans we've tried that in the past (see the discussion thread), but it
doesn't feel like a workable path going forward. There's no new development
happening on the Statefun project, and the current PMC and committers are
focussing elsewhere when it comes to the multiple Flink projects that
exist.

Best regards,

Martijn

On Wed, Jan 28, 2026 at 1:37 PM <fr...@sbbsystems.com> wrote:

> Hi,
>
> I did some work recently on a fork to uplift to Flink 2.2 (migrated from
> DataSet to DataStream etc).
>
>
> https://github.com/fransking/flink-statefun/commit/25b2dfcc3feafd701eb2fec32d413fab56e1cfe6
>
> I did drop kenesis because I have no way to test it.  I'd be happy to
> contribute the changes back but I'd need some help from the community
> navigating the process.
>
> Thanks,
>
> Frans
>
> -----Original Message-----
> From: Galen Warren via dev <de...@flink.apache.org>
> Sent: 28 January 2026 14:14
> To: Flink Dev <de...@flink.apache.org>
> Cc: user <us...@flink.apache.org>; Galen Warren <ga...@cvillewarrens.com>
> Subject: Re: [DISCUSS] Status of Statefun Project
>
> I had been using it - and contributed some code - but have given up on it
> and moved on to other solutions since it's clearly not being supported.
>
> On Wed, Jan 28, 2026, 6:30 AM Martijn Visser <ma...@apache.org>
> wrote:
>
> > I forgot to include the link to this discussion thread for those that
> > don't have the history, that can be found at
> > https://lists.apache.org/thread/7cr2bgt91ppk6pz8o0nfbd10gs63nz6t
> >
> > On Wed, Jan 28, 2026 at 12:28 PM Martijn Visser
> > <ma...@apache.org>
> > wrote:
> >
> > > Hi everyone,
> > >
> > > I'm bringing back this very old thread. It's been a while since we
> > > last discussed this, so I wanted to follow up on where things stand
> > > with Stateful Functions.
> > >
> > >  Since the 3.3.0 release in September 2023, there has been:
> > >  - Zero commits to the repository
> > >  - No new releases (now 4+ Flink versions behind: 1.16 vs 1.20/2.2)
> > >  - 22 open PRs unreviewed
> > >  - No sustained contributor activity
> > >
> > > I don't think that the current Flink committers and PMC members have
> > > the bandwidth or expertise in StateFun's codebase to maintain it.
> > > StateFun was always a specialized subproject maintained by a small
> > > group of dedicated contributors who have since moved on to other
> > > projects.
> > >
> > > Given this situation, I'd like to get the community's input:
> > >
> > > 1. Is anyone still actively using StateFun in production?
> > > 2. Should we consider sunsetting the project rather than leaving it
> > > in limbo?
> > >
> > > Thanks,
> > >
> > > Martijn
> > >
> > > Op wo 23 aug 2023 om 08:48 schreef Filip Karnicki <
> > > filip.karnicki@gmail.com>:
> > > >
> > > > Hi Gordon
> > > >
> > > > Any chance we could get this reviewed and released to the central
> repo?
> > > > We're currently forced to use a Flink version that has a nasty bug
> > > causing
> > > > an operational nightmare
> > > >
> > > > Many thanks
> > > > Fil
> > > >
> > > > On Sat, 19 Aug 2023 at 01:38, Galen Warren via user <
> > > user@flink.apache.org>
> > > > wrote:
> > > >
> > > > > Gotcha, makes sense as to the original division.
> > > > >
> > > > > >> Can this be solved by simply passing in the path to the
> > > > > >> artifacts
> > > > >
> > > > > This definitely works if we're going to be copying the artifacts
> > > > > on
> > the
> > > > > host side -- into the build context -- and then from the context
> > > > > into
> > > the
> > > > > image. It only gets tricky to have a potentially varying path to
> > > > > the artifacts if we're trying to *directly *include the
> > > > > artifacts in the Docker context -- then we have a situation
> > > > > where the Docker context
> > > must
> > > > > contain both the artifacts and playground files, with
> > > > > (potentially) different root locations.
> > > > >
> > > > > Maybe the simplest thing to do here is just to leave the
> > > > > playground
> > > as-is
> > > > > and then copy the artifacts into the Docker context manually,
> > > > > prior
> > to
> > > > > building the playground images. I'm fine with that. It will mean
> > > > > that
> > > each
> > > > > Statefun release will require two PRs and two sets of
> > > > > build/publish
> > > steps
> > > > > instead of one, but if everyone else is fine with that I am, too.
> > > Unless
> > > > > anyone objects, I'll go ahead and queue up a PR for the
> > > > > playground
> > that
> > > > > makes these changes.
> > > > >
> > > > > Also, I should mention -- in case it's not clear -- that I have
> > already
> > > > > built and run the playground examples with the code from the PR
> > > > > and everything worked. So that PR is ready to move forward with
> > > > > review,
> > > etc.,
> > > > > at this point.
> > > > >
> > > > > Thanks.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Fri, Aug 18, 2023 at 4:16 PM Tzu-Li (Gordon) Tai <
> > > tzulitai@apache.org>
> > > > > wrote:
> > > > >
> > > > >> Hi Galen,
> > > > >>
> > > > >> The original intent of having a separate repo for the
> > > > >> playground
> > repo,
> > > > >> was that StateFun users can just go to that and start running
> > > > >> simple examples without any other distractions from the core
> > > > >> code. I
> > > personally
> > > > >> don't have a strong preference here and can understand how it
> > > > >> would
> > > make
> > > > >> the workflow more streamlined, but just FYI on the reasoning
> > > > >> why are separate in the first place.
> > > > >>
> > > > >> re: paths for locating StateFun artifacts.
> > > > >> Can this be solved by simply passing in the path to the artifacts?
> > As
> > > > >> well as the image tag for the locally build base StateFun image.
> > They
> > > could
> > > > >> probably be environment variables.
> > > > >>
> > > > >> Cheers,
> > > > >> Gordon
> > > > >>
> > > > >> On Fri, Aug 18, 2023 at 12:13 PM Galen Warren via user <
> > > > >> user@flink.apache.org> wrote:
> > > > >>
> > > > >>> Yes, exactly! And in addition to the base Statefun jars and
> > > > >>> the jar
> > > for
> > > > >>> the Java SDK, it does an equivalent copy/register operation
> > > > >>> for
> > each
> > > of the
> > > > >>> other SDK libraries (Go, Python, Javascript) so that those
> > libraries
> > > are
> > > > >>> also available when building the playground examples.
> > > > >>>
> > > > >>> One more question: In order to copy the various build
> > > > >>> artifacts
> > into
> > > the
> > > > >>> Docker containers, those artifacts need to be part of the
> > > > >>> Docker
> > > context.
> > > > >>> With the playground being a separate project, that's slightly
> > tricky
> > > to do,
> > > > >>> as there is no guarantee (other than convention) about the
> > > > >>> relative
> > > paths
> > > > >>> of *flink-statefun* and* flink-statefun-playground *in
> > > > >>> someone's
> > > local
> > > > >>> filesystem. The way I've set this up locally is to copy the
> > > playground into
> > > > >>> the* flink-statefun* project -- i.e. to
> > > > >>> *flink-statefun*/playground
> > > --
> > > > >>> such that I can set the Docker context to the root folder of
> > > > >>> *flink-statefun* and then have access to any local code and/or
> > build
> > > > >>> artifacts.
> > > > >>>
> > > > >>> I'm wondering if there might be any appetite for making that
> > > > >>> move permanent, i.e. moving the playground to
> > *flink-statefun*/playground
> > > > >>> and deprecating the standalone playground project. In addition
> > > > >>> to
> > > making
> > > > >>> the problem of building with unreleased artifacts a bit
> > > > >>> simpler to
> > > solve,
> > > > >>> it would also simplify the process of releasing a new Statefun
> > > version,
> > > > >>> since the entire process could be handled with a single PR and
> > > associated
> > > > >>> build/deploy tasks. In other words, a single PR could both
> > > > >>> update
> > and
> > > > >>> deploy the Statefun package and the playground code and images.
> > > > >>>
> > > > >>> As it stands, at least two PRs would be required for each
> > > > >>> Statefun version update -- one for *flink-statefun* and one
> > > > >>> for *flink-statefun-playground*.
> > > > >>>
> > > > >>> Anyway, just an idea. Maybe there's an important reason for
> > > > >>> these projects to remain separate. If we do want to keep the
> > > > >>> playground
> > > project
> > > > >>> where it is, I could solve the copying problem by requiring
> > > > >>> the two projects to be siblings in the file system and by
> > > > >>> pre-copying the
> > > local
> > > > >>> build artifacts into a location accessible by the existing
> > > > >>> Docker
> > > contexts.
> > > > >>> This would still leave us with the need to have two PRs and
> > releases
> > > > >>> instead of one, though.
> > > > >>>
> > > > >>> Thanks for your help!
> > > > >>>
> > > > >>>
> > > > >>> On Fri, Aug 18, 2023 at 2:45 PM Tzu-Li (Gordon) Tai <
> > > tzulitai@apache.org>
> > > > >>> wrote:
> > > > >>>
> > > > >>>> Hi Galen,
> > > > >>>>
> > > > >>>> > locally built code is copied into the build containers
> > > > >>>> so that it can be accessed during the build.
> > > > >>>>
> > > > >>>> That's exactly what we had been doing for release testing, yes.
> > > Sorry I
> > > > >>>> missed that detail in my previous response.
> > > > >>>>
> > > > >>>> And yes, that sounds like a reasonable approach. If I
> > > > >>>> understand
> > you
> > > > >>>> correctly, the workflow would become this:
> > > > >>>>
> > > > >>>>    1. Build the StateFun repo locally to install the snapshot
> > > artifact
> > > > >>>>    jars + have a local base StateFun image.
> > > > >>>>    2. Run the playground in "local" mode, so that it uses the
> > local
> > > base
> > > > >>>>    StateFun image + builds the playground code using copied
> > artifact
> > > > >>>> jars
> > > > >>>>    (instead of pulling from Maven).
> > > > >>>>
> > > > >>>> That looks good to me!
> > > > >>>>
> > > > >>>> Thanks,
> > > > >>>> Gordon
> > > > >>>>
> > > > >>>> On Fri, Aug 18, 2023 at 11:33 AM Galen Warren
> > > > >>>> <ga...@cvillewarrens.com.invalid> wrote:
> > > > >>>>
> > > > >>>> > Thanks.
> > > > >>>> >
> > > > >>>> > If you were to build a local image, as you suggest, how do
> > > > >>>> > you
> > > access
> > > > >>>> that
> > > > >>>> > image when building the playground images? All the
> > > > >>>> > compilation
> > of
> > > > >>>> > playground code happens inside containers, so local images
> > > > >>>> > on
> > the
> > > host
> > > > >>>> > aren't available in those containers. Unless I'm missing
> > > something?
> > > > >>>> >
> > > > >>>> > I've slightly reworked things such that the playground
> > > > >>>> > images
> > can
> > > be
> > > > >>>> run in
> > > > >>>> > one of two modes -- the default mode, which works like
> > > > >>>> > before,
> > > and a
> > > > >>>> > "local" mode where locally built code is copied into the
> > > > >>>> > build
> > > > >>>> containers
> > > > >>>> > so that it can be accessed during the build. It works fine,
> > > > >>>> > you
> > > just
> > > > >>>> have
> > > > >>>> > to define a couple of environment variables when running
> > > > >>>> docker-compose to
> > > > >>>> > specify default vs. local mode and what versions of Flink
> > > > >>>> > and
> > > Statefun
> > > > >>>> > should be referenced, and then you can build a run the
> > > > >>>> > local
> > > examples
> > > > >>>> > without any additional steps. Does that sound like a
> > > > >>>> > reasonable
> > > > >>>> approach?
> > > > >>>> >
> > > > >>>> >
> > > > >>>> > On Fri, Aug 18, 2023 at 2:17 PM Tzu-Li (Gordon) Tai <
> > > > >>>> tzulitai@apache.org>
> > > > >>>> > wrote:
> > > > >>>> >
> > > > >>>> > > Hi Galen,
> > > > >>>> > >
> > > > >>>> > > > Gordon, is there a trick to running the sample code in
> > > > >>>> > > flink-statefun-playground against yet-unreleased code
> > > > >>>> > > that I'm
> > > > >>>> missing?
> > > > >>>> > >
> > > > >>>> > > You'd have to locally build an image from the release
> > > > >>>> > > branch,
> > > with a
> > > > >>>> > > temporary image version tag. Then, in the
> > > flink-statefun-playground,
> > > > >>>> > change
> > > > >>>> > > the image versions in the docker-compose files to use
> > > > >>>> > > that
> > > locally
> > > > >>>> built
> > > > >>>> > > image. IIRC, that's what we have been doing in the past.
> > > > >>>> Admittedly, it's
> > > > >>>> > > pretty manual - I don't think the CI manages this workflow.
> > > > >>>> > >
> > > > >>>> > > Thanks,
> > > > >>>> > > Gordon
> > > > >>>> > >
> > > > >>>> > > On Mon, Aug 14, 2023 at 10:42 AM Galen Warren <
> > > > >>>> galen@cvillewarrens.com>
> > > > >>>> > > wrote:
> > > > >>>> > >
> > > > >>>> > > > I created a pull request for this: [FLINK-31619]
> > > > >>>> > > > Upgrade
> > > Stateful
> > > > >>>> > > > Functions to Flink 1.16.1 by galenwarren · Pull Request
> > #331 ·
> > > > >>>> > > > apache/flink-statefun (github.com)
> > > > >>>> > > > <https://github.com/apache/flink-statefun/pull/331>.
> > > > >>>> > > >
> > > > >>>> > > > JIRA is here: [FLINK-31619] Upgrade Stateful Functions
> > > > >>>> > > > to
> > > Flink
> > > > >>>> 1.16.1
> > > > >>>> > -
> > > > >>>> > > > ASF JIRA (apache.org)
> > > > >>>> > > > <
> > https://issues.apache.org/jira/browse/FLINK-31619?filter=-1
> > > >.
> > > > >>>> > > >
> > > > >>>> > > > Statefun references 1.16.2, despite the title -- that
> > version
> > > has
> > > > >>>> come
> > > > >>>> > > out
> > > > >>>> > > > since the issue was created.
> > > > >>>> > > >
> > > > >>>> > > > I figured out how to run all the playground tests
> > > > >>>> > > > locally,
> > > but it
> > > > >>>> took
> > > > >>>> > a
> > > > >>>> > > > bit of reworking of the playground setup with respect
> > > > >>>> > > > to
> > > Docker;
> > > > >>>> > > > specifically, the Docker contexts used to build the
> > > > >>>> > > > example
> > > > >>>> functions
> > > > >>>> > > > needed to be broadened (i.e. moved up the tree) so
> > > > >>>> > > > that, if
> > > > >>>> needed,
> > > > >>>> > local
> > > > >>>> > > > artifacts/code can be accessed from within the
> > > > >>>> > > > containers at
> > > build
> > > > >>>> > time.
> > > > >>>> > > > Then I made the Docker compose.yml configurable through
> > > > >>>> environment
> > > > >>>> > > > variables to allow for them to run in either the
> > > > >>>> > > > original
> > > manner
> > > > >>>> --
> > > > >>>> > i.e.
> > > > >>>> > > > pulling artifacts from public repos -- or in a "local"
> > > > >>>> > > > mode,
> > > where
> > > > >>>> > > > artifacts are pulled from local builds.
> > > > >>>> > > >
> > > > >>>> > > > This process is a cleaner if the playground is a
> > > > >>>> > > > subfolder
> > of
> > > the
> > > > >>>> > > > flink-statefun project rather than be its own
> > > > >>>> > > > repository (flink-statefun-playground), because then
> > > > >>>> > > > all the relative
> > > paths
> > > > >>>> > between
> > > > >>>> > > > the playground files and the build artifacts are fixed.
> > > > >>>> > > > So,
> > > I'd
> > > > >>>> like to
> > > > >>>> > > > propose to move the playground files, modified as
> > > > >>>> > > > described
> > > > >>>> above, to
> > > > >>>> > > > flink-statefun/playground and retire
> > > flink-statefun-playground. I
> > > > >>>> can
> > > > >>>> > > > submit separate PR s those changes if everyone is on
> board.
> > > > >>>> > > >
> > > > >>>> > > > Also, should I plan to do the same upgrade to handle
> > > > >>>> > > > Flink
> > > > >>>> 1.17.x? It
> > > > >>>> > > > should be easy to do, especially while the 1.16.x
> > > > >>>> > > > upgrade is
> > > > >>>> fresh on
> > > > >>>> > my
> > > > >>>> > > > mind.
> > > > >>>> > > >
> > > > >>>> > > > Thanks.
> > > > >>>> > > >
> > > > >>>> > > >
> > > > >>>> > > > On Fri, Aug 11, 2023 at 6:40 PM Galen Warren <
> > > > >>>> galen@cvillewarrens.com>
> > > > >>>> > > > wrote:
> > > > >>>> > > >
> > > > >>>> > > >> I'm done with the code to make Statefun compatible
> > > > >>>> > > >> with
> > Flink
> > > > >>>> 1.16,
> > > > >>>> > and
> > > > >>>> > > >> all the tests (including e2e succeed). The required
> > > > >>>> > > >> changes
> > > were
> > > > >>>> > pretty
> > > > >>>> > > >> minimal.
> > > > >>>> > > >>
> > > > >>>> > > >> I'm running into a bit of a chicken/egg problem
> > > > >>>> > > >> executing
> > the
> > > > >>>> tests in
> > > > >>>> > > >> flink-statefun-playground
> > > > >>>> > > >> <https://github.com/apache/flink-statefun-playground>,
> > > though.
> > > > >>>> That
> > > > >>>> > > >> project seems to assume that all the various Statefun
> > > artifacts
> > > > >>>> are
> > > > >>>> > > built
> > > > >>>> > > >> and deployed to the various public repositories already.
> > I've
> > > > >>>> looked
> > > > >>>> > > into
> > > > >>>> > > >> trying to redirect references to local artifacts;
> > > > >>>> > > >> however,
> > > > >>>> that's also
> > > > >>>> > > >> tricky since all the building occurs in Docker
> containers.
> > > > >>>> > > >>
> > > > >>>> > > >> Gordon, is there a trick to running the sample code in
> > > > >>>> > > >> flink-statefun-playground against yet-unreleased code
> > > > >>>> > > >> that
> > > I'm
> > > > >>>> > missing?
> > > > >>>> > > >>
> > > > >>>> > > >> Thanks.
> > > > >>>> > > >>
> > > > >>>> > > >> On Sat, Jun 24, 2023 at 12:40 PM Galen Warren <
> > > > >>>> > galen@cvillewarrens.com>
> > > > >>>> > > >> wrote:
> > > > >>>> > > >>
> > > > >>>> > > >>> Great -- thanks!
> > > > >>>> > > >>>
> > > > >>>> > > >>> I'm going to be out of town for about a week but I'll
> > take a
> > > > >>>> look at
> > > > >>>> > > >>> this when I'm back.
> > > > >>>> > > >>>
> > > > >>>> > > >>> On Tue, Jun 20, 2023 at 8:46 AM Martijn Visser <
> > > > >>>> mvisser@confluent.io
> > > > >>>> > >
> > > > >>>> > > >>> wrote:
> > > > >>>> > > >>>
> > > > >>>> > > >>>> Hi Galen,
> > > > >>>> > > >>>>
> > > > >>>> > > >>>> Yes, I'll be more than happy to help with Statefun
> > > releases.
> > > > >>>> > > >>>>
> > > > >>>> > > >>>> Best regards,
> > > > >>>> > > >>>>
> > > > >>>> > > >>>> Martijn
> > > > >>>> > > >>>>
> > > > >>>> > > >>>> On Tue, Jun 20, 2023 at 2:21 PM Galen Warren <
> > > > >>>> > galen@cvillewarrens.com
> > > > >>>> > > >
> > > > >>>> > > >>>> wrote:
> > > > >>>> > > >>>>
> > > > >>>> > > >>>>> Thanks.
> > > > >>>> > > >>>>>
> > > > >>>> > > >>>>> Martijn, to answer your question, I'd need to do a
> > > > >>>> > > >>>>> small
> > > > >>>> amount of
> > > > >>>> > > >>>>> work to get a PR ready, but not much. Happy to do
> > > > >>>> > > >>>>> it if
> > > we're
> > > > >>>> > > deciding to
> > > > >>>> > > >>>>> restart Statefun releases -- are we?
> > > > >>>> > > >>>>>
> > > > >>>> > > >>>>> -- Galen
> > > > >>>> > > >>>>>
> > > > >>>> > > >>>>> On Sat, Jun 17, 2023 at 9:47 AM Tzu-Li (Gordon) Tai
> > > > >>>> > > >>>>> < tzulitai@apache.org> wrote:
> > > > >>>> > > >>>>>
> > > > >>>> > > >>>>>> > Perhaps he could weigh in on whether the
> > > > >>>> > > >>>>>> > combination
> > of
> > > > >>>> > automated
> > > > >>>> > > >>>>>> tests plus those smoke tests should be sufficient
> > > > >>>> > > >>>>>> for
> > > > >>>> testing with
> > > > >>>> > > new
> > > > >>>> > > >>>>>> Flink versions
> > > > >>>> > > >>>>>>
> > > > >>>> > > >>>>>> What we usually did at the bare minimum for new
> > StateFun
> > > > >>>> releases
> > > > >>>> > > was
> > > > >>>> > > >>>>>> the following:
> > > > >>>> > > >>>>>>
> > > > >>>> > > >>>>>>    1. Build tests (including the smoke tests in
> > > > >>>> > > >>>>>> the e2e
> > > > >>>> module,
> > > > >>>> > > >>>>>>    which covers important tests like exactly-once
> > > > >>>> verification)
> > > > >>>> > > >>>>>>    2. Updating the flink-statefun-playground repo
> > > > >>>> > > >>>>>> and
> > > > >>>> manually
> > > > >>>> > > >>>>>>    running all language examples there.
> > > > >>>> > > >>>>>>
> > > > >>>> > > >>>>>> If upgrading Flink versions was the only change in
> > > > >>>> > > >>>>>> the
> > > > >>>> release,
> > > > >>>> > I'd
> > > > >>>> > > >>>>>> probably say that this is sufficient.
> > > > >>>> > > >>>>>>
> > > > >>>> > > >>>>>> Best,
> > > > >>>> > > >>>>>> Gordon
> > > > >>>> > > >>>>>>
> > > > >>>> > > >>>>>> On Thu, Jun 15, 2023 at 5:25 AM Martijn Visser <
> > > > >>>> > > >>>>>> martijnvisser@apache.org> wrote:
> > > > >>>> > > >>>>>>
> > > > >>>> > > >>>>>>> Let me know if you have a PR for a Flink update
> > > > >>>> > > >>>>>>> :)
> > > > >>>> > > >>>>>>>
> > > > >>>> > > >>>>>>> On Thu, Jun 8, 2023 at 5:52 PM Galen Warren via
> > > > >>>> > > >>>>>>> user < user@flink.apache.org> wrote:
> > > > >>>> > > >>>>>>>
> > > > >>>> > > >>>>>>>> Thanks Martijn.
> > > > >>>> > > >>>>>>>>
> > > > >>>> > > >>>>>>>> Personally, I'm already using a local fork of
> > Statefun
> > > > >>>> that is
> > > > >>>> > > >>>>>>>> compatible with Flink 1.16.x, so I wouldn't have
> > > > >>>> > > >>>>>>>> any
> > > need
> > > > >>>> for a
> > > > >>>> > > released
> > > > >>>> > > >>>>>>>> version compatible with 1.15.x. I'd be happy to
> > > > >>>> > > >>>>>>>> do
> > the
> > > PRs
> > > > >>>> to
> > > > >>>> > > modify
> > > > >>>> > > >>>>>>>> Statefun to work with new versions of Flink as
> > > > >>>> > > >>>>>>>> they
> > > come
> > > > >>>> along.
> > > > >>>> > > >>>>>>>>
> > > > >>>> > > >>>>>>>> As for testing, Statefun does have unit tests
> > > > >>>> > > >>>>>>>> and
> > > Gordon
> > > > >>>> also
> > > > >>>> > sent
> > > > >>>> > > >>>>>>>> me instructions a while back for how to do some
> > > additional
> > > > >>>> smoke
> > > > >>>> > > tests
> > > > >>>> > > >>>>>>>> which are pretty straightforward. Perhaps he
> > > > >>>> > > >>>>>>>> could
> > > weigh
> > > > >>>> in on
> > > > >>>> > > whether the
> > > > >>>> > > >>>>>>>> combination of automated tests plus those smoke
> > > > >>>> > > >>>>>>>> tests
> > > > >>>> should be
> > > > >>>> > > sufficient
> > > > >>>> > > >>>>>>>> for testing with new Flink versions (I believe
> > > > >>>> > > >>>>>>>> the
> > > answer
> > > > >>>> is
> > > > >>>> > yes).
> > > > >>>> > > >>>>>>>>
> > > > >>>> > > >>>>>>>> -- Galen
> > > > >>>> > > >>>>>>>>
> > > > >>>> > > >>>>>>>>
> > > > >>>> > > >>>>>>>>
> > > > >>>> > > >>>>>>>> On Thu, Jun 8, 2023 at 8:01 AM Martijn Visser <
> > > > >>>> > > >>>>>>>> martijnvisser@apache.org> wrote:
> > > > >>>> > > >>>>>>>>
> > > > >>>> > > >>>>>>>>> Hi all,
> > > > >>>> > > >>>>>>>>>
> > > > >>>> > > >>>>>>>>> Apologies for the late reply.
> > > > >>>> > > >>>>>>>>>
> > > > >>>> > > >>>>>>>>> I'm willing to help out with merging requests
> > > > >>>> > > >>>>>>>>> in
> > > Statefun
> > > > >>>> to
> > > > >>>> > keep
> > > > >>>> > > >>>>>>>>> them
> > > > >>>> > > >>>>>>>>> compatible with new Flink releases and create
> > > > >>>> > > >>>>>>>>> new
> > > > >>>> releases. I
> > > > >>>> > do
> > > > >>>> > > >>>>>>>>> think that
> > > > >>>> > > >>>>>>>>> validation of the functionality of these
> > > > >>>> > > >>>>>>>>> releases
> > > depends
> > > > >>>> a lot
> > > > >>>> > > on
> > > > >>>> > > >>>>>>>>> those
> > > > >>>> > > >>>>>>>>> who do these compatibility updates, with PMC
> > > > >>>> > > >>>>>>>>> members
> > > > >>>> helping
> > > > >>>> > out
> > > > >>>> > > >>>>>>>>> with the
> > > > >>>> > > >>>>>>>>> formal process.
> > > > >>>> > > >>>>>>>>>
> > > > >>>> > > >>>>>>>>> > Why can't the Apache Software Foundation
> > > > >>>> > > >>>>>>>>> > allow
> > > community
> > > > >>>> > > members
> > > > >>>> > > >>>>>>>>> to bring
> > > > >>>> > > >>>>>>>>> it up to date?
> > > > >>>> > > >>>>>>>>>
> > > > >>>> > > >>>>>>>>> There's nothing preventing anyone from
> > > > >>>> > > >>>>>>>>> reviewing any
> > > of
> > > > >>>> the
> > > > >>>> > > >>>>>>>>> current PRs or
> > > > >>>> > > >>>>>>>>> opening new ones. However, none of them are
> > > > >>>> > > >>>>>>>>> approved
> > > [1],
> > > > >>>> so
> > > > >>>> > > >>>>>>>>> there's also
> > > > >>>> > > >>>>>>>>> nothing to merge.
> > > > >>>> > > >>>>>>>>>
> > > > >>>> > > >>>>>>>>> > I believe that there are people and companies
> > > > >>>> > > >>>>>>>>> > on
> > > this
> > > > >>>> mailing
> > > > >>>> > > >>>>>>>>> list
> > > > >>>> > > >>>>>>>>> interested in supporting Apache Flink Stateful
> > > Functions.
> > > > >>>> > > >>>>>>>>>
> > > > >>>> > > >>>>>>>>> If so, then now is the time to show.
> > > > >>>> > > >>>>>>>>>
> > > > >>>> > > >>>>>>>>> Would there be a preference to create a release
> > > > >>>> > > >>>>>>>>> with
> > > > >>>> Galen's
> > > > >>>> > > merged
> > > > >>>> > > >>>>>>>>> compatibility update to Flink 1.15.2, or do we
> > > > >>>> > > >>>>>>>>> want
> > to
> > > > >>>> skip
> > > > >>>> > that
> > > > >>>> > > >>>>>>>>> and go
> > > > >>>> > > >>>>>>>>> straight to a newer version?
> > > > >>>> > > >>>>>>>>>
> > > > >>>> > > >>>>>>>>> Best regards,
> > > > >>>> > > >>>>>>>>>
> > > > >>>> > > >>>>>>>>> Martijn
> > > > >>>> > > >>>>>>>>>
> > > > >>>> > > >>>>>>>>> [1]
> > > > >>>> > > >>>>>>>>>
> > > > >>>> > > >>>>>>>>>
> > > > >>>> > >
> > > > >>>> >
> > > > >>>>
> > >
> > https://github.com/apache/flink-statefun/pulls?q=is%3Apr+is%3Aopen+rev
> > iew%3Aapproved
> > > > >>>> > > >>>>>>>>>
> > > > >>>> > > >>>>>>>>> On Tue, Jun 6, 2023 at 3:55 PM Marco Villalobos
> > > > >>>> > > >>>>>>>>> < mvillalobos@kineteque.com>
> > > > >>>> > > >>>>>>>>> wrote:
> > > > >>>> > > >>>>>>>>>
> > > > >>>> > > >>>>>>>>> > Why can't the Apache Software Foundation
> > > > >>>> > > >>>>>>>>> > allow
> > > community
> > > > >>>> > > members
> > > > >>>> > > >>>>>>>>> to bring
> > > > >>>> > > >>>>>>>>> > it up to date?
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > What's the process for that?
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > I believe that there are people and companies
> > > > >>>> > > >>>>>>>>> > on
> > > this
> > > > >>>> mailing
> > > > >>>> > > >>>>>>>>> list
> > > > >>>> > > >>>>>>>>> > interested in supporting Apache Flink
> > > > >>>> > > >>>>>>>>> > Stateful
> > > > >>>> Functions.
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > You already had two people on this thread
> > > > >>>> > > >>>>>>>>> > express
> > > > >>>> interest.
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > At the very least, we could keep the library
> > > versions
> > > > >>>> up to
> > > > >>>> > > date.
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > There are only a small list of new features
> > > > >>>> > > >>>>>>>>> > that
> > > might
> > > > >>>> be
> > > > >>>> > > >>>>>>>>> worthwhile:
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > 1. event time processing 2. state rest api
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <
> > > > >>>> > > chesnay@apache.org>
> > > > >>>> > > >>>>>>>>> wrote:
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > If you were to fork it *and want to redistribute
> > it*
> > > > >>>> then the
> > > > >>>> > > >>>>>>>>> short
> > > > >>>> > > >>>>>>>>> > version is that
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> >    1. you have to adhere to the Apache licensing
> > > > >>>> requirements
> > > > >>>> > > >>>>>>>>> >    2. you have to make it clear that your fork
> > does
> > > not
> > > > >>>> > belong
> > > > >>>> > > >>>>>>>>> to the
> > > > >>>> > > >>>>>>>>> >    Apache Flink project. (Trademarks and all
> that)
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > Neither should be significant hurdles (there
> > should
> > > > >>>> also be
> > > > >>>> > > >>>>>>>>> plenty of
> > > > >>>> > > >>>>>>>>> > online resources regarding 1), and if you do
> this
> > > then
> > > > >>>> you
> > > > >>>> > can
> > > > >>>> > > >>>>>>>>> freely share
> > > > >>>> > > >>>>>>>>> > your fork with others.
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > I've also pinged Martijn to take a look at this
> > > thread.
> > > > >>>> > > >>>>>>>>> > To my knowledge the project hasn't decided
> > anything
> > > yet.
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > On 27/05/2023 04:05, Galen Warren wrote:
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > Ok, I get it. No interest.
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > If this project is being abandoned, I guess I'll
> > > work
> > > > >>>> with my
> > > > >>>> > > >>>>>>>>> own fork. Is
> > > > >>>> > > >>>>>>>>> > there anything I should consider here? Can I
> share
> > > it
> > > > >>>> with
> > > > >>>> > > other
> > > > >>>> > > >>>>>>>>> people who
> > > > >>>> > > >>>>>>>>> > use this project?
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > On Tue, May 16, 2023 at 10:50 AM Galen Warren <
> > > > >>>> > > >>>>>>>>> galen@cvillewarrens.com> <galen@cvillewarrens.com
> >
> > > > >>>> > > >>>>>>>>> > wrote:
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > Hi Martijn, since you opened this discussion
> > > thread, I'm
> > > > >>>> > > curious
> > > > >>>> > > >>>>>>>>> what your
> > > > >>>> > > >>>>>>>>> > thoughts are in light of the responses? Thanks.
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <
> > > > >>>> > > >>>>>>>>> galen@cvillewarrens.com> <galen@cvillewarrens.com
> >
> > > > >>>> > > >>>>>>>>> > wrote:
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > I use Apache Flink for stream processing, and
> > > StateFun
> > > > >>>> as a
> > > > >>>> > > >>>>>>>>> hand-off
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > point for the rest of the application.
> > > > >>>> > > >>>>>>>>> > It serves well as a bridge between a Flink
> > Streaming
> > > > >>>> job and
> > > > >>>> > > >>>>>>>>> > micro-services.
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > This is essentially how I use it as well, and I
> > > would
> > > > >>>> also be
> > > > >>>> > > >>>>>>>>> sad to see
> > > > >>>> > > >>>>>>>>> > it sunsetted. It works well; I don't know that
> > > there is
> > > > >>>> a lot
> > > > >>>> > > of
> > > > >>>> > > >>>>>>>>> new
> > > > >>>> > > >>>>>>>>> > development required, but if there are no new
> > > Statefun
> > > > >>>> > > releases,
> > > > >>>> > > >>>>>>>>> then
> > > > >>>> > > >>>>>>>>> > Statefun can only be used with older Flink
> > versions.
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > On Tue, Apr 18, 2023 at 10:04 PM Marco
> Villalobos
> > <
> > > > >>>> > > >>>>>>>>> mvillalobos@kineteque.com> wrote:
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > I am currently using Stateful Functions in my
> > > > >>>> application.
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > I use Apache Flink for stream processing, and
> > > StateFun
> > > > >>>> as a
> > > > >>>> > > >>>>>>>>> hand-off
> > > > >>>> > > >>>>>>>>> > point for the rest of the application.
> > > > >>>> > > >>>>>>>>> > It serves well as a bridge between a Flink
> > Streaming
> > > > >>>> job and
> > > > >>>> > > >>>>>>>>> > micro-services.
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > I would be disappointed if StateFun was
> sunsetted.
> > > Its
> > > > >>>> a
> > > > >>>> > good
> > > > >>>> > > >>>>>>>>> idea.
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > If there is anything I can do to help, as a
> > > contributor
> > > > >>>> > > perhaps,
> > > > >>>> > > >>>>>>>>> please
> > > > >>>> > > >>>>>>>>> > let me know.
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <
> > > > >>>> > > >>>>>>>>> martijnvisser@apache.org> <
> martijnvisser@apache.org
> > >
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > wrote:
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > Hi everyone,
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > I want to open a discussion on the status of the
> > > > >>>> Statefun
> > > > >>>> > > >>>>>>>>> Project [1]
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > in Apache Flink. As you might have noticed,
> there
> > > > >>>> hasn't been
> > > > >>>> > > >>>>>>>>> much
> > > > >>>> > > >>>>>>>>> > development over the past months in the Statefun
> > > > >>>> repository
> > > > >>>> > > [2].
> > > > >>>> > > >>>>>>>>> There is
> > > > >>>> > > >>>>>>>>> > currently a lack of active contributors and
> > > committers
> > > > >>>> who
> > > > >>>> > are
> > > > >>>> > > >>>>>>>>> able to help
> > > > >>>> > > >>>>>>>>> > with the maintenance of the project.
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > In order to improve the situation, we need to
> > solve
> > > the
> > > > >>>> lack
> > > > >>>> > of
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > committers and the lack of contributors.
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > On the lack of committers:
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > 1. Ideally, there are some of the current Flink
> > > > >>>> committers
> > > > >>>> > who
> > > > >>>> > > >>>>>>>>> have
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > the bandwidth and can help with reviewing PRs
> and
> > > > >>>> merging
> > > > >>>> > them.
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > 2. If that's not an option, it could be a
> > > consideration
> > > > >>>> that
> > > > >>>> > > >>>>>>>>> current
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > committers only approve and review PRs, that are
> > > > >>>> approved by
> > > > >>>> > > >>>>>>>>> those who are
> > > > >>>> > > >>>>>>>>> > willing to contribute to Statefun and if the CI
> > > passes
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > On the lack of contributors:
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > 3. Next to having this discussion on the Dev and
> > > User
> > > > >>>> mailing
> > > > >>>> > > >>>>>>>>> list, we
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > can also create a blog with a call for new
> > > contributors
> > > > >>>> on
> > > > >>>> > the
> > > > >>>> > > >>>>>>>>> Flink
> > > > >>>> > > >>>>>>>>> > project website, send out some tweets on the
> > Flink /
> > > > >>>> Statefun
> > > > >>>> > > >>>>>>>>> twitter
> > > > >>>> > > >>>>>>>>> > accounts, post messages on Slack etc. In that
> > > message,
> > > > >>>> we
> > > > >>>> > would
> > > > >>>> > > >>>>>>>>> inform how
> > > > >>>> > > >>>>>>>>> > those that are interested in contributing can
> > start
> > > and
> > > > >>>> where
> > > > >>>> > > >>>>>>>>> they could
> > > > >>>> > > >>>>>>>>> > reach out for more information.
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > There's also option 4. where a group of
> interested
> > > > >>>> people
> > > > >>>> > would
> > > > >>>> > > >>>>>>>>> split
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > Statefun from the Flink project and make it a
> > > separate
> > > > >>>> top
> > > > >>>> > > level
> > > > >>>> > > >>>>>>>>> project
> > > > >>>> > > >>>>>>>>> > under the Apache Flink umbrella (similar as
> > > recently has
> > > > >>>> > > >>>>>>>>> happened with
> > > > >>>> > > >>>>>>>>> > Flink Table Store, which has become Apache
> > Paimon).
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > If we see no improvements in the coming period,
> we
> > > > >>>> should
> > > > >>>> > > >>>>>>>>> consider
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > sunsetting Statefun and communicate that clearly
> > to
> > > the
> > > > >>>> > users.
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > I'm looking forward to your thoughts.
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > Best regards,
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > Martijn
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > [1]
> > > > >>>> > > >>>>>>>>>
> > > > >>>> https://nightlies.apache.org/flink/flink-statefun-docs-master/
> > > > >>>> > <
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> >
> https://nightlies.apache.org/flink/flink-statefun-docs-master/
> > > > >>>> > > >
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > [2] https://github.com/apache/flink-statefun <
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > https://github.com/apache/flink-statefun>
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>>
> > > > >>>> > > >>>>>>>>
> > > > >>>> > >
> > > > >>>> >
> > > > >>>>
> > > > >>>
> > >
> >
>
>
>

Re: [DISCUSS] Status of Statefun Project

[Martijn Visser <ma...@apache.org>]

Hi all,

@Gale thanks for sharing that insight.

@Frans we've tried that in the past (see the discussion thread), but it
doesn't feel like a workable path going forward. There's no new development
happening on the Statefun project, and the current PMC and committers are
focussing elsewhere when it comes to the multiple Flink projects that
exist.

Best regards,

Martijn

On Wed, Jan 28, 2026 at 1:37 PM <fr...@sbbsystems.com> wrote:

> Hi,
>
> I did some work recently on a fork to uplift to Flink 2.2 (migrated from
> DataSet to DataStream etc).
>
>
> https://github.com/fransking/flink-statefun/commit/25b2dfcc3feafd701eb2fec32d413fab56e1cfe6
>
> I did drop kenesis because I have no way to test it.  I'd be happy to
> contribute the changes back but I'd need some help from the community
> navigating the process.
>
> Thanks,
>
> Frans
>
> -----Original Message-----
> From: Galen Warren via dev <de...@flink.apache.org>
> Sent: 28 January 2026 14:14
> To: Flink Dev <de...@flink.apache.org>
> Cc: user <us...@flink.apache.org>; Galen Warren <ga...@cvillewarrens.com>
> Subject: Re: [DISCUSS] Status of Statefun Project
>
> I had been using it - and contributed some code - but have given up on it
> and moved on to other solutions since it's clearly not being supported.
>
> On Wed, Jan 28, 2026, 6:30 AM Martijn Visser <ma...@apache.org>
> wrote:
>
> > I forgot to include the link to this discussion thread for those that
> > don't have the history, that can be found at
> > https://lists.apache.org/thread/7cr2bgt91ppk6pz8o0nfbd10gs63nz6t
> >
> > On Wed, Jan 28, 2026 at 12:28 PM Martijn Visser
> > <ma...@apache.org>
> > wrote:
> >
> > > Hi everyone,
> > >
> > > I'm bringing back this very old thread. It's been a while since we
> > > last discussed this, so I wanted to follow up on where things stand
> > > with Stateful Functions.
> > >
> > >  Since the 3.3.0 release in September 2023, there has been:
> > >  - Zero commits to the repository
> > >  - No new releases (now 4+ Flink versions behind: 1.16 vs 1.20/2.2)
> > >  - 22 open PRs unreviewed
> > >  - No sustained contributor activity
> > >
> > > I don't think that the current Flink committers and PMC members have
> > > the bandwidth or expertise in StateFun's codebase to maintain it.
> > > StateFun was always a specialized subproject maintained by a small
> > > group of dedicated contributors who have since moved on to other
> > > projects.
> > >
> > > Given this situation, I'd like to get the community's input:
> > >
> > > 1. Is anyone still actively using StateFun in production?
> > > 2. Should we consider sunsetting the project rather than leaving it
> > > in limbo?
> > >
> > > Thanks,
> > >
> > > Martijn
> > >
> > > Op wo 23 aug 2023 om 08:48 schreef Filip Karnicki <
> > > filip.karnicki@gmail.com>:
> > > >
> > > > Hi Gordon
> > > >
> > > > Any chance we could get this reviewed and released to the central
> repo?
> > > > We're currently forced to use a Flink version that has a nasty bug
> > > causing
> > > > an operational nightmare
> > > >
> > > > Many thanks
> > > > Fil
> > > >
> > > > On Sat, 19 Aug 2023 at 01:38, Galen Warren via user <
> > > user@flink.apache.org>
> > > > wrote:
> > > >
> > > > > Gotcha, makes sense as to the original division.
> > > > >
> > > > > >> Can this be solved by simply passing in the path to the
> > > > > >> artifacts
> > > > >
> > > > > This definitely works if we're going to be copying the artifacts
> > > > > on
> > the
> > > > > host side -- into the build context -- and then from the context
> > > > > into
> > > the
> > > > > image. It only gets tricky to have a potentially varying path to
> > > > > the artifacts if we're trying to *directly *include the
> > > > > artifacts in the Docker context -- then we have a situation
> > > > > where the Docker context
> > > must
> > > > > contain both the artifacts and playground files, with
> > > > > (potentially) different root locations.
> > > > >
> > > > > Maybe the simplest thing to do here is just to leave the
> > > > > playground
> > > as-is
> > > > > and then copy the artifacts into the Docker context manually,
> > > > > prior
> > to
> > > > > building the playground images. I'm fine with that. It will mean
> > > > > that
> > > each
> > > > > Statefun release will require two PRs and two sets of
> > > > > build/publish
> > > steps
> > > > > instead of one, but if everyone else is fine with that I am, too.
> > > Unless
> > > > > anyone objects, I'll go ahead and queue up a PR for the
> > > > > playground
> > that
> > > > > makes these changes.
> > > > >
> > > > > Also, I should mention -- in case it's not clear -- that I have
> > already
> > > > > built and run the playground examples with the code from the PR
> > > > > and everything worked. So that PR is ready to move forward with
> > > > > review,
> > > etc.,
> > > > > at this point.
> > > > >
> > > > > Thanks.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Fri, Aug 18, 2023 at 4:16 PM Tzu-Li (Gordon) Tai <
> > > tzulitai@apache.org>
> > > > > wrote:
> > > > >
> > > > >> Hi Galen,
> > > > >>
> > > > >> The original intent of having a separate repo for the
> > > > >> playground
> > repo,
> > > > >> was that StateFun users can just go to that and start running
> > > > >> simple examples without any other distractions from the core
> > > > >> code. I
> > > personally
> > > > >> don't have a strong preference here and can understand how it
> > > > >> would
> > > make
> > > > >> the workflow more streamlined, but just FYI on the reasoning
> > > > >> why are separate in the first place.
> > > > >>
> > > > >> re: paths for locating StateFun artifacts.
> > > > >> Can this be solved by simply passing in the path to the artifacts?
> > As
> > > > >> well as the image tag for the locally build base StateFun image.
> > They
> > > could
> > > > >> probably be environment variables.
> > > > >>
> > > > >> Cheers,
> > > > >> Gordon
> > > > >>
> > > > >> On Fri, Aug 18, 2023 at 12:13 PM Galen Warren via user <
> > > > >> user@flink.apache.org> wrote:
> > > > >>
> > > > >>> Yes, exactly! And in addition to the base Statefun jars and
> > > > >>> the jar
> > > for
> > > > >>> the Java SDK, it does an equivalent copy/register operation
> > > > >>> for
> > each
> > > of the
> > > > >>> other SDK libraries (Go, Python, Javascript) so that those
> > libraries
> > > are
> > > > >>> also available when building the playground examples.
> > > > >>>
> > > > >>> One more question: In order to copy the various build
> > > > >>> artifacts
> > into
> > > the
> > > > >>> Docker containers, those artifacts need to be part of the
> > > > >>> Docker
> > > context.
> > > > >>> With the playground being a separate project, that's slightly
> > tricky
> > > to do,
> > > > >>> as there is no guarantee (other than convention) about the
> > > > >>> relative
> > > paths
> > > > >>> of *flink-statefun* and* flink-statefun-playground *in
> > > > >>> someone's
> > > local
> > > > >>> filesystem. The way I've set this up locally is to copy the
> > > playground into
> > > > >>> the* flink-statefun* project -- i.e. to
> > > > >>> *flink-statefun*/playground
> > > --
> > > > >>> such that I can set the Docker context to the root folder of
> > > > >>> *flink-statefun* and then have access to any local code and/or
> > build
> > > > >>> artifacts.
> > > > >>>
> > > > >>> I'm wondering if there might be any appetite for making that
> > > > >>> move permanent, i.e. moving the playground to
> > *flink-statefun*/playground
> > > > >>> and deprecating the standalone playground project. In addition
> > > > >>> to
> > > making
> > > > >>> the problem of building with unreleased artifacts a bit
> > > > >>> simpler to
> > > solve,
> > > > >>> it would also simplify the process of releasing a new Statefun
> > > version,
> > > > >>> since the entire process could be handled with a single PR and
> > > associated
> > > > >>> build/deploy tasks. In other words, a single PR could both
> > > > >>> update
> > and
> > > > >>> deploy the Statefun package and the playground code and images.
> > > > >>>
> > > > >>> As it stands, at least two PRs would be required for each
> > > > >>> Statefun version update -- one for *flink-statefun* and one
> > > > >>> for *flink-statefun-playground*.
> > > > >>>
> > > > >>> Anyway, just an idea. Maybe there's an important reason for
> > > > >>> these projects to remain separate. If we do want to keep the
> > > > >>> playground
> > > project
> > > > >>> where it is, I could solve the copying problem by requiring
> > > > >>> the two projects to be siblings in the file system and by
> > > > >>> pre-copying the
> > > local
> > > > >>> build artifacts into a location accessible by the existing
> > > > >>> Docker
> > > contexts.
> > > > >>> This would still leave us with the need to have two PRs and
> > releases
> > > > >>> instead of one, though.
> > > > >>>
> > > > >>> Thanks for your help!
> > > > >>>
> > > > >>>
> > > > >>> On Fri, Aug 18, 2023 at 2:45 PM Tzu-Li (Gordon) Tai <
> > > tzulitai@apache.org>
> > > > >>> wrote:
> > > > >>>
> > > > >>>> Hi Galen,
> > > > >>>>
> > > > >>>> > locally built code is copied into the build containers
> > > > >>>> so that it can be accessed during the build.
> > > > >>>>
> > > > >>>> That's exactly what we had been doing for release testing, yes.
> > > Sorry I
> > > > >>>> missed that detail in my previous response.
> > > > >>>>
> > > > >>>> And yes, that sounds like a reasonable approach. If I
> > > > >>>> understand
> > you
> > > > >>>> correctly, the workflow would become this:
> > > > >>>>
> > > > >>>>    1. Build the StateFun repo locally to install the snapshot
> > > artifact
> > > > >>>>    jars + have a local base StateFun image.
> > > > >>>>    2. Run the playground in "local" mode, so that it uses the
> > local
> > > base
> > > > >>>>    StateFun image + builds the playground code using copied
> > artifact
> > > > >>>> jars
> > > > >>>>    (instead of pulling from Maven).
> > > > >>>>
> > > > >>>> That looks good to me!
> > > > >>>>
> > > > >>>> Thanks,
> > > > >>>> Gordon
> > > > >>>>
> > > > >>>> On Fri, Aug 18, 2023 at 11:33 AM Galen Warren
> > > > >>>> <ga...@cvillewarrens.com.invalid> wrote:
> > > > >>>>
> > > > >>>> > Thanks.
> > > > >>>> >
> > > > >>>> > If you were to build a local image, as you suggest, how do
> > > > >>>> > you
> > > access
> > > > >>>> that
> > > > >>>> > image when building the playground images? All the
> > > > >>>> > compilation
> > of
> > > > >>>> > playground code happens inside containers, so local images
> > > > >>>> > on
> > the
> > > host
> > > > >>>> > aren't available in those containers. Unless I'm missing
> > > something?
> > > > >>>> >
> > > > >>>> > I've slightly reworked things such that the playground
> > > > >>>> > images
> > can
> > > be
> > > > >>>> run in
> > > > >>>> > one of two modes -- the default mode, which works like
> > > > >>>> > before,
> > > and a
> > > > >>>> > "local" mode where locally built code is copied into the
> > > > >>>> > build
> > > > >>>> containers
> > > > >>>> > so that it can be accessed during the build. It works fine,
> > > > >>>> > you
> > > just
> > > > >>>> have
> > > > >>>> > to define a couple of environment variables when running
> > > > >>>> docker-compose to
> > > > >>>> > specify default vs. local mode and what versions of Flink
> > > > >>>> > and
> > > Statefun
> > > > >>>> > should be referenced, and then you can build a run the
> > > > >>>> > local
> > > examples
> > > > >>>> > without any additional steps. Does that sound like a
> > > > >>>> > reasonable
> > > > >>>> approach?
> > > > >>>> >
> > > > >>>> >
> > > > >>>> > On Fri, Aug 18, 2023 at 2:17 PM Tzu-Li (Gordon) Tai <
> > > > >>>> tzulitai@apache.org>
> > > > >>>> > wrote:
> > > > >>>> >
> > > > >>>> > > Hi Galen,
> > > > >>>> > >
> > > > >>>> > > > Gordon, is there a trick to running the sample code in
> > > > >>>> > > flink-statefun-playground against yet-unreleased code
> > > > >>>> > > that I'm
> > > > >>>> missing?
> > > > >>>> > >
> > > > >>>> > > You'd have to locally build an image from the release
> > > > >>>> > > branch,
> > > with a
> > > > >>>> > > temporary image version tag. Then, in the
> > > flink-statefun-playground,
> > > > >>>> > change
> > > > >>>> > > the image versions in the docker-compose files to use
> > > > >>>> > > that
> > > locally
> > > > >>>> built
> > > > >>>> > > image. IIRC, that's what we have been doing in the past.
> > > > >>>> Admittedly, it's
> > > > >>>> > > pretty manual - I don't think the CI manages this workflow.
> > > > >>>> > >
> > > > >>>> > > Thanks,
> > > > >>>> > > Gordon
> > > > >>>> > >
> > > > >>>> > > On Mon, Aug 14, 2023 at 10:42 AM Galen Warren <
> > > > >>>> galen@cvillewarrens.com>
> > > > >>>> > > wrote:
> > > > >>>> > >
> > > > >>>> > > > I created a pull request for this: [FLINK-31619]
> > > > >>>> > > > Upgrade
> > > Stateful
> > > > >>>> > > > Functions to Flink 1.16.1 by galenwarren · Pull Request
> > #331 ·
> > > > >>>> > > > apache/flink-statefun (github.com)
> > > > >>>> > > > <https://github.com/apache/flink-statefun/pull/331>.
> > > > >>>> > > >
> > > > >>>> > > > JIRA is here: [FLINK-31619] Upgrade Stateful Functions
> > > > >>>> > > > to
> > > Flink
> > > > >>>> 1.16.1
> > > > >>>> > -
> > > > >>>> > > > ASF JIRA (apache.org)
> > > > >>>> > > > <
> > https://issues.apache.org/jira/browse/FLINK-31619?filter=-1
> > > >.
> > > > >>>> > > >
> > > > >>>> > > > Statefun references 1.16.2, despite the title -- that
> > version
> > > has
> > > > >>>> come
> > > > >>>> > > out
> > > > >>>> > > > since the issue was created.
> > > > >>>> > > >
> > > > >>>> > > > I figured out how to run all the playground tests
> > > > >>>> > > > locally,
> > > but it
> > > > >>>> took
> > > > >>>> > a
> > > > >>>> > > > bit of reworking of the playground setup with respect
> > > > >>>> > > > to
> > > Docker;
> > > > >>>> > > > specifically, the Docker contexts used to build the
> > > > >>>> > > > example
> > > > >>>> functions
> > > > >>>> > > > needed to be broadened (i.e. moved up the tree) so
> > > > >>>> > > > that, if
> > > > >>>> needed,
> > > > >>>> > local
> > > > >>>> > > > artifacts/code can be accessed from within the
> > > > >>>> > > > containers at
> > > build
> > > > >>>> > time.
> > > > >>>> > > > Then I made the Docker compose.yml configurable through
> > > > >>>> environment
> > > > >>>> > > > variables to allow for them to run in either the
> > > > >>>> > > > original
> > > manner
> > > > >>>> --
> > > > >>>> > i.e.
> > > > >>>> > > > pulling artifacts from public repos -- or in a "local"
> > > > >>>> > > > mode,
> > > where
> > > > >>>> > > > artifacts are pulled from local builds.
> > > > >>>> > > >
> > > > >>>> > > > This process is a cleaner if the playground is a
> > > > >>>> > > > subfolder
> > of
> > > the
> > > > >>>> > > > flink-statefun project rather than be its own
> > > > >>>> > > > repository (flink-statefun-playground), because then
> > > > >>>> > > > all the relative
> > > paths
> > > > >>>> > between
> > > > >>>> > > > the playground files and the build artifacts are fixed.
> > > > >>>> > > > So,
> > > I'd
> > > > >>>> like to
> > > > >>>> > > > propose to move the playground files, modified as
> > > > >>>> > > > described
> > > > >>>> above, to
> > > > >>>> > > > flink-statefun/playground and retire
> > > flink-statefun-playground. I
> > > > >>>> can
> > > > >>>> > > > submit separate PR s those changes if everyone is on
> board.
> > > > >>>> > > >
> > > > >>>> > > > Also, should I plan to do the same upgrade to handle
> > > > >>>> > > > Flink
> > > > >>>> 1.17.x? It
> > > > >>>> > > > should be easy to do, especially while the 1.16.x
> > > > >>>> > > > upgrade is
> > > > >>>> fresh on
> > > > >>>> > my
> > > > >>>> > > > mind.
> > > > >>>> > > >
> > > > >>>> > > > Thanks.
> > > > >>>> > > >
> > > > >>>> > > >
> > > > >>>> > > > On Fri, Aug 11, 2023 at 6:40 PM Galen Warren <
> > > > >>>> galen@cvillewarrens.com>
> > > > >>>> > > > wrote:
> > > > >>>> > > >
> > > > >>>> > > >> I'm done with the code to make Statefun compatible
> > > > >>>> > > >> with
> > Flink
> > > > >>>> 1.16,
> > > > >>>> > and
> > > > >>>> > > >> all the tests (including e2e succeed). The required
> > > > >>>> > > >> changes
> > > were
> > > > >>>> > pretty
> > > > >>>> > > >> minimal.
> > > > >>>> > > >>
> > > > >>>> > > >> I'm running into a bit of a chicken/egg problem
> > > > >>>> > > >> executing
> > the
> > > > >>>> tests in
> > > > >>>> > > >> flink-statefun-playground
> > > > >>>> > > >> <https://github.com/apache/flink-statefun-playground>,
> > > though.
> > > > >>>> That
> > > > >>>> > > >> project seems to assume that all the various Statefun
> > > artifacts
> > > > >>>> are
> > > > >>>> > > built
> > > > >>>> > > >> and deployed to the various public repositories already.
> > I've
> > > > >>>> looked
> > > > >>>> > > into
> > > > >>>> > > >> trying to redirect references to local artifacts;
> > > > >>>> > > >> however,
> > > > >>>> that's also
> > > > >>>> > > >> tricky since all the building occurs in Docker
> containers.
> > > > >>>> > > >>
> > > > >>>> > > >> Gordon, is there a trick to running the sample code in
> > > > >>>> > > >> flink-statefun-playground against yet-unreleased code
> > > > >>>> > > >> that
> > > I'm
> > > > >>>> > missing?
> > > > >>>> > > >>
> > > > >>>> > > >> Thanks.
> > > > >>>> > > >>
> > > > >>>> > > >> On Sat, Jun 24, 2023 at 12:40 PM Galen Warren <
> > > > >>>> > galen@cvillewarrens.com>
> > > > >>>> > > >> wrote:
> > > > >>>> > > >>
> > > > >>>> > > >>> Great -- thanks!
> > > > >>>> > > >>>
> > > > >>>> > > >>> I'm going to be out of town for about a week but I'll
> > take a
> > > > >>>> look at
> > > > >>>> > > >>> this when I'm back.
> > > > >>>> > > >>>
> > > > >>>> > > >>> On Tue, Jun 20, 2023 at 8:46 AM Martijn Visser <
> > > > >>>> mvisser@confluent.io
> > > > >>>> > >
> > > > >>>> > > >>> wrote:
> > > > >>>> > > >>>
> > > > >>>> > > >>>> Hi Galen,
> > > > >>>> > > >>>>
> > > > >>>> > > >>>> Yes, I'll be more than happy to help with Statefun
> > > releases.
> > > > >>>> > > >>>>
> > > > >>>> > > >>>> Best regards,
> > > > >>>> > > >>>>
> > > > >>>> > > >>>> Martijn
> > > > >>>> > > >>>>
> > > > >>>> > > >>>> On Tue, Jun 20, 2023 at 2:21 PM Galen Warren <
> > > > >>>> > galen@cvillewarrens.com
> > > > >>>> > > >
> > > > >>>> > > >>>> wrote:
> > > > >>>> > > >>>>
> > > > >>>> > > >>>>> Thanks.
> > > > >>>> > > >>>>>
> > > > >>>> > > >>>>> Martijn, to answer your question, I'd need to do a
> > > > >>>> > > >>>>> small
> > > > >>>> amount of
> > > > >>>> > > >>>>> work to get a PR ready, but not much. Happy to do
> > > > >>>> > > >>>>> it if
> > > we're
> > > > >>>> > > deciding to
> > > > >>>> > > >>>>> restart Statefun releases -- are we?
> > > > >>>> > > >>>>>
> > > > >>>> > > >>>>> -- Galen
> > > > >>>> > > >>>>>
> > > > >>>> > > >>>>> On Sat, Jun 17, 2023 at 9:47 AM Tzu-Li (Gordon) Tai
> > > > >>>> > > >>>>> < tzulitai@apache.org> wrote:
> > > > >>>> > > >>>>>
> > > > >>>> > > >>>>>> > Perhaps he could weigh in on whether the
> > > > >>>> > > >>>>>> > combination
> > of
> > > > >>>> > automated
> > > > >>>> > > >>>>>> tests plus those smoke tests should be sufficient
> > > > >>>> > > >>>>>> for
> > > > >>>> testing with
> > > > >>>> > > new
> > > > >>>> > > >>>>>> Flink versions
> > > > >>>> > > >>>>>>
> > > > >>>> > > >>>>>> What we usually did at the bare minimum for new
> > StateFun
> > > > >>>> releases
> > > > >>>> > > was
> > > > >>>> > > >>>>>> the following:
> > > > >>>> > > >>>>>>
> > > > >>>> > > >>>>>>    1. Build tests (including the smoke tests in
> > > > >>>> > > >>>>>> the e2e
> > > > >>>> module,
> > > > >>>> > > >>>>>>    which covers important tests like exactly-once
> > > > >>>> verification)
> > > > >>>> > > >>>>>>    2. Updating the flink-statefun-playground repo
> > > > >>>> > > >>>>>> and
> > > > >>>> manually
> > > > >>>> > > >>>>>>    running all language examples there.
> > > > >>>> > > >>>>>>
> > > > >>>> > > >>>>>> If upgrading Flink versions was the only change in
> > > > >>>> > > >>>>>> the
> > > > >>>> release,
> > > > >>>> > I'd
> > > > >>>> > > >>>>>> probably say that this is sufficient.
> > > > >>>> > > >>>>>>
> > > > >>>> > > >>>>>> Best,
> > > > >>>> > > >>>>>> Gordon
> > > > >>>> > > >>>>>>
> > > > >>>> > > >>>>>> On Thu, Jun 15, 2023 at 5:25 AM Martijn Visser <
> > > > >>>> > > >>>>>> martijnvisser@apache.org> wrote:
> > > > >>>> > > >>>>>>
> > > > >>>> > > >>>>>>> Let me know if you have a PR for a Flink update
> > > > >>>> > > >>>>>>> :)
> > > > >>>> > > >>>>>>>
> > > > >>>> > > >>>>>>> On Thu, Jun 8, 2023 at 5:52 PM Galen Warren via
> > > > >>>> > > >>>>>>> user < user@flink.apache.org> wrote:
> > > > >>>> > > >>>>>>>
> > > > >>>> > > >>>>>>>> Thanks Martijn.
> > > > >>>> > > >>>>>>>>
> > > > >>>> > > >>>>>>>> Personally, I'm already using a local fork of
> > Statefun
> > > > >>>> that is
> > > > >>>> > > >>>>>>>> compatible with Flink 1.16.x, so I wouldn't have
> > > > >>>> > > >>>>>>>> any
> > > need
> > > > >>>> for a
> > > > >>>> > > released
> > > > >>>> > > >>>>>>>> version compatible with 1.15.x. I'd be happy to
> > > > >>>> > > >>>>>>>> do
> > the
> > > PRs
> > > > >>>> to
> > > > >>>> > > modify
> > > > >>>> > > >>>>>>>> Statefun to work with new versions of Flink as
> > > > >>>> > > >>>>>>>> they
> > > come
> > > > >>>> along.
> > > > >>>> > > >>>>>>>>
> > > > >>>> > > >>>>>>>> As for testing, Statefun does have unit tests
> > > > >>>> > > >>>>>>>> and
> > > Gordon
> > > > >>>> also
> > > > >>>> > sent
> > > > >>>> > > >>>>>>>> me instructions a while back for how to do some
> > > additional
> > > > >>>> smoke
> > > > >>>> > > tests
> > > > >>>> > > >>>>>>>> which are pretty straightforward. Perhaps he
> > > > >>>> > > >>>>>>>> could
> > > weigh
> > > > >>>> in on
> > > > >>>> > > whether the
> > > > >>>> > > >>>>>>>> combination of automated tests plus those smoke
> > > > >>>> > > >>>>>>>> tests
> > > > >>>> should be
> > > > >>>> > > sufficient
> > > > >>>> > > >>>>>>>> for testing with new Flink versions (I believe
> > > > >>>> > > >>>>>>>> the
> > > answer
> > > > >>>> is
> > > > >>>> > yes).
> > > > >>>> > > >>>>>>>>
> > > > >>>> > > >>>>>>>> -- Galen
> > > > >>>> > > >>>>>>>>
> > > > >>>> > > >>>>>>>>
> > > > >>>> > > >>>>>>>>
> > > > >>>> > > >>>>>>>> On Thu, Jun 8, 2023 at 8:01 AM Martijn Visser <
> > > > >>>> > > >>>>>>>> martijnvisser@apache.org> wrote:
> > > > >>>> > > >>>>>>>>
> > > > >>>> > > >>>>>>>>> Hi all,
> > > > >>>> > > >>>>>>>>>
> > > > >>>> > > >>>>>>>>> Apologies for the late reply.
> > > > >>>> > > >>>>>>>>>
> > > > >>>> > > >>>>>>>>> I'm willing to help out with merging requests
> > > > >>>> > > >>>>>>>>> in
> > > Statefun
> > > > >>>> to
> > > > >>>> > keep
> > > > >>>> > > >>>>>>>>> them
> > > > >>>> > > >>>>>>>>> compatible with new Flink releases and create
> > > > >>>> > > >>>>>>>>> new
> > > > >>>> releases. I
> > > > >>>> > do
> > > > >>>> > > >>>>>>>>> think that
> > > > >>>> > > >>>>>>>>> validation of the functionality of these
> > > > >>>> > > >>>>>>>>> releases
> > > depends
> > > > >>>> a lot
> > > > >>>> > > on
> > > > >>>> > > >>>>>>>>> those
> > > > >>>> > > >>>>>>>>> who do these compatibility updates, with PMC
> > > > >>>> > > >>>>>>>>> members
> > > > >>>> helping
> > > > >>>> > out
> > > > >>>> > > >>>>>>>>> with the
> > > > >>>> > > >>>>>>>>> formal process.
> > > > >>>> > > >>>>>>>>>
> > > > >>>> > > >>>>>>>>> > Why can't the Apache Software Foundation
> > > > >>>> > > >>>>>>>>> > allow
> > > community
> > > > >>>> > > members
> > > > >>>> > > >>>>>>>>> to bring
> > > > >>>> > > >>>>>>>>> it up to date?
> > > > >>>> > > >>>>>>>>>
> > > > >>>> > > >>>>>>>>> There's nothing preventing anyone from
> > > > >>>> > > >>>>>>>>> reviewing any
> > > of
> > > > >>>> the
> > > > >>>> > > >>>>>>>>> current PRs or
> > > > >>>> > > >>>>>>>>> opening new ones. However, none of them are
> > > > >>>> > > >>>>>>>>> approved
> > > [1],
> > > > >>>> so
> > > > >>>> > > >>>>>>>>> there's also
> > > > >>>> > > >>>>>>>>> nothing to merge.
> > > > >>>> > > >>>>>>>>>
> > > > >>>> > > >>>>>>>>> > I believe that there are people and companies
> > > > >>>> > > >>>>>>>>> > on
> > > this
> > > > >>>> mailing
> > > > >>>> > > >>>>>>>>> list
> > > > >>>> > > >>>>>>>>> interested in supporting Apache Flink Stateful
> > > Functions.
> > > > >>>> > > >>>>>>>>>
> > > > >>>> > > >>>>>>>>> If so, then now is the time to show.
> > > > >>>> > > >>>>>>>>>
> > > > >>>> > > >>>>>>>>> Would there be a preference to create a release
> > > > >>>> > > >>>>>>>>> with
> > > > >>>> Galen's
> > > > >>>> > > merged
> > > > >>>> > > >>>>>>>>> compatibility update to Flink 1.15.2, or do we
> > > > >>>> > > >>>>>>>>> want
> > to
> > > > >>>> skip
> > > > >>>> > that
> > > > >>>> > > >>>>>>>>> and go
> > > > >>>> > > >>>>>>>>> straight to a newer version?
> > > > >>>> > > >>>>>>>>>
> > > > >>>> > > >>>>>>>>> Best regards,
> > > > >>>> > > >>>>>>>>>
> > > > >>>> > > >>>>>>>>> Martijn
> > > > >>>> > > >>>>>>>>>
> > > > >>>> > > >>>>>>>>> [1]
> > > > >>>> > > >>>>>>>>>
> > > > >>>> > > >>>>>>>>>
> > > > >>>> > >
> > > > >>>> >
> > > > >>>>
> > >
> > https://github.com/apache/flink-statefun/pulls?q=is%3Apr+is%3Aopen+rev
> > iew%3Aapproved
> > > > >>>> > > >>>>>>>>>
> > > > >>>> > > >>>>>>>>> On Tue, Jun 6, 2023 at 3:55 PM Marco Villalobos
> > > > >>>> > > >>>>>>>>> < mvillalobos@kineteque.com>
> > > > >>>> > > >>>>>>>>> wrote:
> > > > >>>> > > >>>>>>>>>
> > > > >>>> > > >>>>>>>>> > Why can't the Apache Software Foundation
> > > > >>>> > > >>>>>>>>> > allow
> > > community
> > > > >>>> > > members
> > > > >>>> > > >>>>>>>>> to bring
> > > > >>>> > > >>>>>>>>> > it up to date?
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > What's the process for that?
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > I believe that there are people and companies
> > > > >>>> > > >>>>>>>>> > on
> > > this
> > > > >>>> mailing
> > > > >>>> > > >>>>>>>>> list
> > > > >>>> > > >>>>>>>>> > interested in supporting Apache Flink
> > > > >>>> > > >>>>>>>>> > Stateful
> > > > >>>> Functions.
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > You already had two people on this thread
> > > > >>>> > > >>>>>>>>> > express
> > > > >>>> interest.
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > At the very least, we could keep the library
> > > versions
> > > > >>>> up to
> > > > >>>> > > date.
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > There are only a small list of new features
> > > > >>>> > > >>>>>>>>> > that
> > > might
> > > > >>>> be
> > > > >>>> > > >>>>>>>>> worthwhile:
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > 1. event time processing 2. state rest api
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <
> > > > >>>> > > chesnay@apache.org>
> > > > >>>> > > >>>>>>>>> wrote:
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > If you were to fork it *and want to redistribute
> > it*
> > > > >>>> then the
> > > > >>>> > > >>>>>>>>> short
> > > > >>>> > > >>>>>>>>> > version is that
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> >    1. you have to adhere to the Apache licensing
> > > > >>>> requirements
> > > > >>>> > > >>>>>>>>> >    2. you have to make it clear that your fork
> > does
> > > not
> > > > >>>> > belong
> > > > >>>> > > >>>>>>>>> to the
> > > > >>>> > > >>>>>>>>> >    Apache Flink project. (Trademarks and all
> that)
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > Neither should be significant hurdles (there
> > should
> > > > >>>> also be
> > > > >>>> > > >>>>>>>>> plenty of
> > > > >>>> > > >>>>>>>>> > online resources regarding 1), and if you do
> this
> > > then
> > > > >>>> you
> > > > >>>> > can
> > > > >>>> > > >>>>>>>>> freely share
> > > > >>>> > > >>>>>>>>> > your fork with others.
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > I've also pinged Martijn to take a look at this
> > > thread.
> > > > >>>> > > >>>>>>>>> > To my knowledge the project hasn't decided
> > anything
> > > yet.
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > On 27/05/2023 04:05, Galen Warren wrote:
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > Ok, I get it. No interest.
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > If this project is being abandoned, I guess I'll
> > > work
> > > > >>>> with my
> > > > >>>> > > >>>>>>>>> own fork. Is
> > > > >>>> > > >>>>>>>>> > there anything I should consider here? Can I
> share
> > > it
> > > > >>>> with
> > > > >>>> > > other
> > > > >>>> > > >>>>>>>>> people who
> > > > >>>> > > >>>>>>>>> > use this project?
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > On Tue, May 16, 2023 at 10:50 AM Galen Warren <
> > > > >>>> > > >>>>>>>>> galen@cvillewarrens.com> <galen@cvillewarrens.com
> >
> > > > >>>> > > >>>>>>>>> > wrote:
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > Hi Martijn, since you opened this discussion
> > > thread, I'm
> > > > >>>> > > curious
> > > > >>>> > > >>>>>>>>> what your
> > > > >>>> > > >>>>>>>>> > thoughts are in light of the responses? Thanks.
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <
> > > > >>>> > > >>>>>>>>> galen@cvillewarrens.com> <galen@cvillewarrens.com
> >
> > > > >>>> > > >>>>>>>>> > wrote:
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > I use Apache Flink for stream processing, and
> > > StateFun
> > > > >>>> as a
> > > > >>>> > > >>>>>>>>> hand-off
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > point for the rest of the application.
> > > > >>>> > > >>>>>>>>> > It serves well as a bridge between a Flink
> > Streaming
> > > > >>>> job and
> > > > >>>> > > >>>>>>>>> > micro-services.
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > This is essentially how I use it as well, and I
> > > would
> > > > >>>> also be
> > > > >>>> > > >>>>>>>>> sad to see
> > > > >>>> > > >>>>>>>>> > it sunsetted. It works well; I don't know that
> > > there is
> > > > >>>> a lot
> > > > >>>> > > of
> > > > >>>> > > >>>>>>>>> new
> > > > >>>> > > >>>>>>>>> > development required, but if there are no new
> > > Statefun
> > > > >>>> > > releases,
> > > > >>>> > > >>>>>>>>> then
> > > > >>>> > > >>>>>>>>> > Statefun can only be used with older Flink
> > versions.
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > On Tue, Apr 18, 2023 at 10:04 PM Marco
> Villalobos
> > <
> > > > >>>> > > >>>>>>>>> mvillalobos@kineteque.com> wrote:
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > I am currently using Stateful Functions in my
> > > > >>>> application.
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > I use Apache Flink for stream processing, and
> > > StateFun
> > > > >>>> as a
> > > > >>>> > > >>>>>>>>> hand-off
> > > > >>>> > > >>>>>>>>> > point for the rest of the application.
> > > > >>>> > > >>>>>>>>> > It serves well as a bridge between a Flink
> > Streaming
> > > > >>>> job and
> > > > >>>> > > >>>>>>>>> > micro-services.
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > I would be disappointed if StateFun was
> sunsetted.
> > > Its
> > > > >>>> a
> > > > >>>> > good
> > > > >>>> > > >>>>>>>>> idea.
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > If there is anything I can do to help, as a
> > > contributor
> > > > >>>> > > perhaps,
> > > > >>>> > > >>>>>>>>> please
> > > > >>>> > > >>>>>>>>> > let me know.
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <
> > > > >>>> > > >>>>>>>>> martijnvisser@apache.org> <
> martijnvisser@apache.org
> > >
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > wrote:
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > Hi everyone,
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > I want to open a discussion on the status of the
> > > > >>>> Statefun
> > > > >>>> > > >>>>>>>>> Project [1]
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > in Apache Flink. As you might have noticed,
> there
> > > > >>>> hasn't been
> > > > >>>> > > >>>>>>>>> much
> > > > >>>> > > >>>>>>>>> > development over the past months in the Statefun
> > > > >>>> repository
> > > > >>>> > > [2].
> > > > >>>> > > >>>>>>>>> There is
> > > > >>>> > > >>>>>>>>> > currently a lack of active contributors and
> > > committers
> > > > >>>> who
> > > > >>>> > are
> > > > >>>> > > >>>>>>>>> able to help
> > > > >>>> > > >>>>>>>>> > with the maintenance of the project.
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > In order to improve the situation, we need to
> > solve
> > > the
> > > > >>>> lack
> > > > >>>> > of
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > committers and the lack of contributors.
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > On the lack of committers:
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > 1. Ideally, there are some of the current Flink
> > > > >>>> committers
> > > > >>>> > who
> > > > >>>> > > >>>>>>>>> have
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > the bandwidth and can help with reviewing PRs
> and
> > > > >>>> merging
> > > > >>>> > them.
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > 2. If that's not an option, it could be a
> > > consideration
> > > > >>>> that
> > > > >>>> > > >>>>>>>>> current
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > committers only approve and review PRs, that are
> > > > >>>> approved by
> > > > >>>> > > >>>>>>>>> those who are
> > > > >>>> > > >>>>>>>>> > willing to contribute to Statefun and if the CI
> > > passes
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > On the lack of contributors:
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > 3. Next to having this discussion on the Dev and
> > > User
> > > > >>>> mailing
> > > > >>>> > > >>>>>>>>> list, we
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > can also create a blog with a call for new
> > > contributors
> > > > >>>> on
> > > > >>>> > the
> > > > >>>> > > >>>>>>>>> Flink
> > > > >>>> > > >>>>>>>>> > project website, send out some tweets on the
> > Flink /
> > > > >>>> Statefun
> > > > >>>> > > >>>>>>>>> twitter
> > > > >>>> > > >>>>>>>>> > accounts, post messages on Slack etc. In that
> > > message,
> > > > >>>> we
> > > > >>>> > would
> > > > >>>> > > >>>>>>>>> inform how
> > > > >>>> > > >>>>>>>>> > those that are interested in contributing can
> > start
> > > and
> > > > >>>> where
> > > > >>>> > > >>>>>>>>> they could
> > > > >>>> > > >>>>>>>>> > reach out for more information.
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > There's also option 4. where a group of
> interested
> > > > >>>> people
> > > > >>>> > would
> > > > >>>> > > >>>>>>>>> split
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > Statefun from the Flink project and make it a
> > > separate
> > > > >>>> top
> > > > >>>> > > level
> > > > >>>> > > >>>>>>>>> project
> > > > >>>> > > >>>>>>>>> > under the Apache Flink umbrella (similar as
> > > recently has
> > > > >>>> > > >>>>>>>>> happened with
> > > > >>>> > > >>>>>>>>> > Flink Table Store, which has become Apache
> > Paimon).
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > If we see no improvements in the coming period,
> we
> > > > >>>> should
> > > > >>>> > > >>>>>>>>> consider
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > sunsetting Statefun and communicate that clearly
> > to
> > > the
> > > > >>>> > users.
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > I'm looking forward to your thoughts.
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > Best regards,
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > Martijn
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > [1]
> > > > >>>> > > >>>>>>>>>
> > > > >>>> https://nightlies.apache.org/flink/flink-statefun-docs-master/
> > > > >>>> > <
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> >
> https://nightlies.apache.org/flink/flink-statefun-docs-master/
> > > > >>>> > > >
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > [2] https://github.com/apache/flink-statefun <
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> > https://github.com/apache/flink-statefun>
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>> >
> > > > >>>> > > >>>>>>>>>
> > > > >>>> > > >>>>>>>>
> > > > >>>> > >
> > > > >>>> >
> > > > >>>>
> > > > >>>
> > >
> >
>
>
>

RE: [DISCUSS] Status of Statefun Project

[fr...@sbbsystems.com]

Hi,

I did some work recently on a fork to uplift to Flink 2.2 (migrated from DataSet to DataStream etc).

https://github.com/fransking/flink-statefun/commit/25b2dfcc3feafd701eb2fec32d413fab56e1cfe6

I did drop kenesis because I have no way to test it.  I'd be happy to contribute the changes back but I'd need some help from the community navigating the process.

Thanks,

Frans

-----Original Message-----
From: Galen Warren via dev <de...@flink.apache.org> 
Sent: 28 January 2026 14:14
To: Flink Dev <de...@flink.apache.org>
Cc: user <us...@flink.apache.org>; Galen Warren <ga...@cvillewarrens.com>
Subject: Re: [DISCUSS] Status of Statefun Project

I had been using it - and contributed some code - but have given up on it and moved on to other solutions since it's clearly not being supported.

On Wed, Jan 28, 2026, 6:30 AM Martijn Visser <ma...@apache.org>
wrote:

> I forgot to include the link to this discussion thread for those that 
> don't have the history, that can be found at 
> https://lists.apache.org/thread/7cr2bgt91ppk6pz8o0nfbd10gs63nz6t
>
> On Wed, Jan 28, 2026 at 12:28 PM Martijn Visser 
> <ma...@apache.org>
> wrote:
>
> > Hi everyone,
> >
> > I'm bringing back this very old thread. It's been a while since we 
> > last discussed this, so I wanted to follow up on where things stand 
> > with Stateful Functions.
> >
> >  Since the 3.3.0 release in September 2023, there has been:
> >  - Zero commits to the repository
> >  - No new releases (now 4+ Flink versions behind: 1.16 vs 1.20/2.2)
> >  - 22 open PRs unreviewed
> >  - No sustained contributor activity
> >
> > I don't think that the current Flink committers and PMC members have 
> > the bandwidth or expertise in StateFun's codebase to maintain it.
> > StateFun was always a specialized subproject maintained by a small 
> > group of dedicated contributors who have since moved on to other 
> > projects.
> >
> > Given this situation, I'd like to get the community's input:
> >
> > 1. Is anyone still actively using StateFun in production?
> > 2. Should we consider sunsetting the project rather than leaving it 
> > in limbo?
> >
> > Thanks,
> >
> > Martijn
> >
> > Op wo 23 aug 2023 om 08:48 schreef Filip Karnicki <
> > filip.karnicki@gmail.com>:
> > >
> > > Hi Gordon
> > >
> > > Any chance we could get this reviewed and released to the central repo?
> > > We're currently forced to use a Flink version that has a nasty bug
> > causing
> > > an operational nightmare
> > >
> > > Many thanks
> > > Fil
> > >
> > > On Sat, 19 Aug 2023 at 01:38, Galen Warren via user <
> > user@flink.apache.org>
> > > wrote:
> > >
> > > > Gotcha, makes sense as to the original division.
> > > >
> > > > >> Can this be solved by simply passing in the path to the 
> > > > >> artifacts
> > > >
> > > > This definitely works if we're going to be copying the artifacts 
> > > > on
> the
> > > > host side -- into the build context -- and then from the context 
> > > > into
> > the
> > > > image. It only gets tricky to have a potentially varying path to 
> > > > the artifacts if we're trying to *directly *include the 
> > > > artifacts in the Docker context -- then we have a situation 
> > > > where the Docker context
> > must
> > > > contain both the artifacts and playground files, with 
> > > > (potentially) different root locations.
> > > >
> > > > Maybe the simplest thing to do here is just to leave the 
> > > > playground
> > as-is
> > > > and then copy the artifacts into the Docker context manually, 
> > > > prior
> to
> > > > building the playground images. I'm fine with that. It will mean 
> > > > that
> > each
> > > > Statefun release will require two PRs and two sets of 
> > > > build/publish
> > steps
> > > > instead of one, but if everyone else is fine with that I am, too.
> > Unless
> > > > anyone objects, I'll go ahead and queue up a PR for the 
> > > > playground
> that
> > > > makes these changes.
> > > >
> > > > Also, I should mention -- in case it's not clear -- that I have
> already
> > > > built and run the playground examples with the code from the PR 
> > > > and everything worked. So that PR is ready to move forward with 
> > > > review,
> > etc.,
> > > > at this point.
> > > >
> > > > Thanks.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On Fri, Aug 18, 2023 at 4:16 PM Tzu-Li (Gordon) Tai <
> > tzulitai@apache.org>
> > > > wrote:
> > > >
> > > >> Hi Galen,
> > > >>
> > > >> The original intent of having a separate repo for the 
> > > >> playground
> repo,
> > > >> was that StateFun users can just go to that and start running 
> > > >> simple examples without any other distractions from the core 
> > > >> code. I
> > personally
> > > >> don't have a strong preference here and can understand how it 
> > > >> would
> > make
> > > >> the workflow more streamlined, but just FYI on the reasoning 
> > > >> why are separate in the first place.
> > > >>
> > > >> re: paths for locating StateFun artifacts.
> > > >> Can this be solved by simply passing in the path to the artifacts?
> As
> > > >> well as the image tag for the locally build base StateFun image.
> They
> > could
> > > >> probably be environment variables.
> > > >>
> > > >> Cheers,
> > > >> Gordon
> > > >>
> > > >> On Fri, Aug 18, 2023 at 12:13 PM Galen Warren via user < 
> > > >> user@flink.apache.org> wrote:
> > > >>
> > > >>> Yes, exactly! And in addition to the base Statefun jars and 
> > > >>> the jar
> > for
> > > >>> the Java SDK, it does an equivalent copy/register operation 
> > > >>> for
> each
> > of the
> > > >>> other SDK libraries (Go, Python, Javascript) so that those
> libraries
> > are
> > > >>> also available when building the playground examples.
> > > >>>
> > > >>> One more question: In order to copy the various build 
> > > >>> artifacts
> into
> > the
> > > >>> Docker containers, those artifacts need to be part of the 
> > > >>> Docker
> > context.
> > > >>> With the playground being a separate project, that's slightly
> tricky
> > to do,
> > > >>> as there is no guarantee (other than convention) about the 
> > > >>> relative
> > paths
> > > >>> of *flink-statefun* and* flink-statefun-playground *in 
> > > >>> someone's
> > local
> > > >>> filesystem. The way I've set this up locally is to copy the
> > playground into
> > > >>> the* flink-statefun* project -- i.e. to 
> > > >>> *flink-statefun*/playground
> > --
> > > >>> such that I can set the Docker context to the root folder of
> > > >>> *flink-statefun* and then have access to any local code and/or
> build
> > > >>> artifacts.
> > > >>>
> > > >>> I'm wondering if there might be any appetite for making that 
> > > >>> move permanent, i.e. moving the playground to
> *flink-statefun*/playground
> > > >>> and deprecating the standalone playground project. In addition 
> > > >>> to
> > making
> > > >>> the problem of building with unreleased artifacts a bit 
> > > >>> simpler to
> > solve,
> > > >>> it would also simplify the process of releasing a new Statefun
> > version,
> > > >>> since the entire process could be handled with a single PR and
> > associated
> > > >>> build/deploy tasks. In other words, a single PR could both 
> > > >>> update
> and
> > > >>> deploy the Statefun package and the playground code and images.
> > > >>>
> > > >>> As it stands, at least two PRs would be required for each 
> > > >>> Statefun version update -- one for *flink-statefun* and one 
> > > >>> for *flink-statefun-playground*.
> > > >>>
> > > >>> Anyway, just an idea. Maybe there's an important reason for 
> > > >>> these projects to remain separate. If we do want to keep the 
> > > >>> playground
> > project
> > > >>> where it is, I could solve the copying problem by requiring 
> > > >>> the two projects to be siblings in the file system and by 
> > > >>> pre-copying the
> > local
> > > >>> build artifacts into a location accessible by the existing 
> > > >>> Docker
> > contexts.
> > > >>> This would still leave us with the need to have two PRs and
> releases
> > > >>> instead of one, though.
> > > >>>
> > > >>> Thanks for your help!
> > > >>>
> > > >>>
> > > >>> On Fri, Aug 18, 2023 at 2:45 PM Tzu-Li (Gordon) Tai <
> > tzulitai@apache.org>
> > > >>> wrote:
> > > >>>
> > > >>>> Hi Galen,
> > > >>>>
> > > >>>> > locally built code is copied into the build containers
> > > >>>> so that it can be accessed during the build.
> > > >>>>
> > > >>>> That's exactly what we had been doing for release testing, yes.
> > Sorry I
> > > >>>> missed that detail in my previous response.
> > > >>>>
> > > >>>> And yes, that sounds like a reasonable approach. If I 
> > > >>>> understand
> you
> > > >>>> correctly, the workflow would become this:
> > > >>>>
> > > >>>>    1. Build the StateFun repo locally to install the snapshot
> > artifact
> > > >>>>    jars + have a local base StateFun image.
> > > >>>>    2. Run the playground in "local" mode, so that it uses the
> local
> > base
> > > >>>>    StateFun image + builds the playground code using copied
> artifact
> > > >>>> jars
> > > >>>>    (instead of pulling from Maven).
> > > >>>>
> > > >>>> That looks good to me!
> > > >>>>
> > > >>>> Thanks,
> > > >>>> Gordon
> > > >>>>
> > > >>>> On Fri, Aug 18, 2023 at 11:33 AM Galen Warren 
> > > >>>> <ga...@cvillewarrens.com.invalid> wrote:
> > > >>>>
> > > >>>> > Thanks.
> > > >>>> >
> > > >>>> > If you were to build a local image, as you suggest, how do 
> > > >>>> > you
> > access
> > > >>>> that
> > > >>>> > image when building the playground images? All the 
> > > >>>> > compilation
> of
> > > >>>> > playground code happens inside containers, so local images 
> > > >>>> > on
> the
> > host
> > > >>>> > aren't available in those containers. Unless I'm missing
> > something?
> > > >>>> >
> > > >>>> > I've slightly reworked things such that the playground 
> > > >>>> > images
> can
> > be
> > > >>>> run in
> > > >>>> > one of two modes -- the default mode, which works like 
> > > >>>> > before,
> > and a
> > > >>>> > "local" mode where locally built code is copied into the 
> > > >>>> > build
> > > >>>> containers
> > > >>>> > so that it can be accessed during the build. It works fine, 
> > > >>>> > you
> > just
> > > >>>> have
> > > >>>> > to define a couple of environment variables when running
> > > >>>> docker-compose to
> > > >>>> > specify default vs. local mode and what versions of Flink 
> > > >>>> > and
> > Statefun
> > > >>>> > should be referenced, and then you can build a run the 
> > > >>>> > local
> > examples
> > > >>>> > without any additional steps. Does that sound like a 
> > > >>>> > reasonable
> > > >>>> approach?
> > > >>>> >
> > > >>>> >
> > > >>>> > On Fri, Aug 18, 2023 at 2:17 PM Tzu-Li (Gordon) Tai <
> > > >>>> tzulitai@apache.org>
> > > >>>> > wrote:
> > > >>>> >
> > > >>>> > > Hi Galen,
> > > >>>> > >
> > > >>>> > > > Gordon, is there a trick to running the sample code in
> > > >>>> > > flink-statefun-playground against yet-unreleased code 
> > > >>>> > > that I'm
> > > >>>> missing?
> > > >>>> > >
> > > >>>> > > You'd have to locally build an image from the release 
> > > >>>> > > branch,
> > with a
> > > >>>> > > temporary image version tag. Then, in the
> > flink-statefun-playground,
> > > >>>> > change
> > > >>>> > > the image versions in the docker-compose files to use 
> > > >>>> > > that
> > locally
> > > >>>> built
> > > >>>> > > image. IIRC, that's what we have been doing in the past.
> > > >>>> Admittedly, it's
> > > >>>> > > pretty manual - I don't think the CI manages this workflow.
> > > >>>> > >
> > > >>>> > > Thanks,
> > > >>>> > > Gordon
> > > >>>> > >
> > > >>>> > > On Mon, Aug 14, 2023 at 10:42 AM Galen Warren <
> > > >>>> galen@cvillewarrens.com>
> > > >>>> > > wrote:
> > > >>>> > >
> > > >>>> > > > I created a pull request for this: [FLINK-31619] 
> > > >>>> > > > Upgrade
> > Stateful
> > > >>>> > > > Functions to Flink 1.16.1 by galenwarren · Pull Request
> #331 ·
> > > >>>> > > > apache/flink-statefun (github.com) 
> > > >>>> > > > <https://github.com/apache/flink-statefun/pull/331>.
> > > >>>> > > >
> > > >>>> > > > JIRA is here: [FLINK-31619] Upgrade Stateful Functions 
> > > >>>> > > > to
> > Flink
> > > >>>> 1.16.1
> > > >>>> > -
> > > >>>> > > > ASF JIRA (apache.org)
> > > >>>> > > > <
> https://issues.apache.org/jira/browse/FLINK-31619?filter=-1
> > >.
> > > >>>> > > >
> > > >>>> > > > Statefun references 1.16.2, despite the title -- that
> version
> > has
> > > >>>> come
> > > >>>> > > out
> > > >>>> > > > since the issue was created.
> > > >>>> > > >
> > > >>>> > > > I figured out how to run all the playground tests 
> > > >>>> > > > locally,
> > but it
> > > >>>> took
> > > >>>> > a
> > > >>>> > > > bit of reworking of the playground setup with respect 
> > > >>>> > > > to
> > Docker;
> > > >>>> > > > specifically, the Docker contexts used to build the 
> > > >>>> > > > example
> > > >>>> functions
> > > >>>> > > > needed to be broadened (i.e. moved up the tree) so 
> > > >>>> > > > that, if
> > > >>>> needed,
> > > >>>> > local
> > > >>>> > > > artifacts/code can be accessed from within the 
> > > >>>> > > > containers at
> > build
> > > >>>> > time.
> > > >>>> > > > Then I made the Docker compose.yml configurable through
> > > >>>> environment
> > > >>>> > > > variables to allow for them to run in either the 
> > > >>>> > > > original
> > manner
> > > >>>> --
> > > >>>> > i.e.
> > > >>>> > > > pulling artifacts from public repos -- or in a "local" 
> > > >>>> > > > mode,
> > where
> > > >>>> > > > artifacts are pulled from local builds.
> > > >>>> > > >
> > > >>>> > > > This process is a cleaner if the playground is a 
> > > >>>> > > > subfolder
> of
> > the
> > > >>>> > > > flink-statefun project rather than be its own 
> > > >>>> > > > repository (flink-statefun-playground), because then 
> > > >>>> > > > all the relative
> > paths
> > > >>>> > between
> > > >>>> > > > the playground files and the build artifacts are fixed. 
> > > >>>> > > > So,
> > I'd
> > > >>>> like to
> > > >>>> > > > propose to move the playground files, modified as 
> > > >>>> > > > described
> > > >>>> above, to
> > > >>>> > > > flink-statefun/playground and retire
> > flink-statefun-playground. I
> > > >>>> can
> > > >>>> > > > submit separate PR s those changes if everyone is on board.
> > > >>>> > > >
> > > >>>> > > > Also, should I plan to do the same upgrade to handle 
> > > >>>> > > > Flink
> > > >>>> 1.17.x? It
> > > >>>> > > > should be easy to do, especially while the 1.16.x 
> > > >>>> > > > upgrade is
> > > >>>> fresh on
> > > >>>> > my
> > > >>>> > > > mind.
> > > >>>> > > >
> > > >>>> > > > Thanks.
> > > >>>> > > >
> > > >>>> > > >
> > > >>>> > > > On Fri, Aug 11, 2023 at 6:40 PM Galen Warren <
> > > >>>> galen@cvillewarrens.com>
> > > >>>> > > > wrote:
> > > >>>> > > >
> > > >>>> > > >> I'm done with the code to make Statefun compatible 
> > > >>>> > > >> with
> Flink
> > > >>>> 1.16,
> > > >>>> > and
> > > >>>> > > >> all the tests (including e2e succeed). The required 
> > > >>>> > > >> changes
> > were
> > > >>>> > pretty
> > > >>>> > > >> minimal.
> > > >>>> > > >>
> > > >>>> > > >> I'm running into a bit of a chicken/egg problem 
> > > >>>> > > >> executing
> the
> > > >>>> tests in
> > > >>>> > > >> flink-statefun-playground 
> > > >>>> > > >> <https://github.com/apache/flink-statefun-playground>,
> > though.
> > > >>>> That
> > > >>>> > > >> project seems to assume that all the various Statefun
> > artifacts
> > > >>>> are
> > > >>>> > > built
> > > >>>> > > >> and deployed to the various public repositories already.
> I've
> > > >>>> looked
> > > >>>> > > into
> > > >>>> > > >> trying to redirect references to local artifacts; 
> > > >>>> > > >> however,
> > > >>>> that's also
> > > >>>> > > >> tricky since all the building occurs in Docker containers.
> > > >>>> > > >>
> > > >>>> > > >> Gordon, is there a trick to running the sample code in 
> > > >>>> > > >> flink-statefun-playground against yet-unreleased code 
> > > >>>> > > >> that
> > I'm
> > > >>>> > missing?
> > > >>>> > > >>
> > > >>>> > > >> Thanks.
> > > >>>> > > >>
> > > >>>> > > >> On Sat, Jun 24, 2023 at 12:40 PM Galen Warren <
> > > >>>> > galen@cvillewarrens.com>
> > > >>>> > > >> wrote:
> > > >>>> > > >>
> > > >>>> > > >>> Great -- thanks!
> > > >>>> > > >>>
> > > >>>> > > >>> I'm going to be out of town for about a week but I'll
> take a
> > > >>>> look at
> > > >>>> > > >>> this when I'm back.
> > > >>>> > > >>>
> > > >>>> > > >>> On Tue, Jun 20, 2023 at 8:46 AM Martijn Visser <
> > > >>>> mvisser@confluent.io
> > > >>>> > >
> > > >>>> > > >>> wrote:
> > > >>>> > > >>>
> > > >>>> > > >>>> Hi Galen,
> > > >>>> > > >>>>
> > > >>>> > > >>>> Yes, I'll be more than happy to help with Statefun
> > releases.
> > > >>>> > > >>>>
> > > >>>> > > >>>> Best regards,
> > > >>>> > > >>>>
> > > >>>> > > >>>> Martijn
> > > >>>> > > >>>>
> > > >>>> > > >>>> On Tue, Jun 20, 2023 at 2:21 PM Galen Warren <
> > > >>>> > galen@cvillewarrens.com
> > > >>>> > > >
> > > >>>> > > >>>> wrote:
> > > >>>> > > >>>>
> > > >>>> > > >>>>> Thanks.
> > > >>>> > > >>>>>
> > > >>>> > > >>>>> Martijn, to answer your question, I'd need to do a 
> > > >>>> > > >>>>> small
> > > >>>> amount of
> > > >>>> > > >>>>> work to get a PR ready, but not much. Happy to do 
> > > >>>> > > >>>>> it if
> > we're
> > > >>>> > > deciding to
> > > >>>> > > >>>>> restart Statefun releases -- are we?
> > > >>>> > > >>>>>
> > > >>>> > > >>>>> -- Galen
> > > >>>> > > >>>>>
> > > >>>> > > >>>>> On Sat, Jun 17, 2023 at 9:47 AM Tzu-Li (Gordon) Tai 
> > > >>>> > > >>>>> < tzulitai@apache.org> wrote:
> > > >>>> > > >>>>>
> > > >>>> > > >>>>>> > Perhaps he could weigh in on whether the 
> > > >>>> > > >>>>>> > combination
> of
> > > >>>> > automated
> > > >>>> > > >>>>>> tests plus those smoke tests should be sufficient 
> > > >>>> > > >>>>>> for
> > > >>>> testing with
> > > >>>> > > new
> > > >>>> > > >>>>>> Flink versions
> > > >>>> > > >>>>>>
> > > >>>> > > >>>>>> What we usually did at the bare minimum for new
> StateFun
> > > >>>> releases
> > > >>>> > > was
> > > >>>> > > >>>>>> the following:
> > > >>>> > > >>>>>>
> > > >>>> > > >>>>>>    1. Build tests (including the smoke tests in 
> > > >>>> > > >>>>>> the e2e
> > > >>>> module,
> > > >>>> > > >>>>>>    which covers important tests like exactly-once
> > > >>>> verification)
> > > >>>> > > >>>>>>    2. Updating the flink-statefun-playground repo 
> > > >>>> > > >>>>>> and
> > > >>>> manually
> > > >>>> > > >>>>>>    running all language examples there.
> > > >>>> > > >>>>>>
> > > >>>> > > >>>>>> If upgrading Flink versions was the only change in 
> > > >>>> > > >>>>>> the
> > > >>>> release,
> > > >>>> > I'd
> > > >>>> > > >>>>>> probably say that this is sufficient.
> > > >>>> > > >>>>>>
> > > >>>> > > >>>>>> Best,
> > > >>>> > > >>>>>> Gordon
> > > >>>> > > >>>>>>
> > > >>>> > > >>>>>> On Thu, Jun 15, 2023 at 5:25 AM Martijn Visser < 
> > > >>>> > > >>>>>> martijnvisser@apache.org> wrote:
> > > >>>> > > >>>>>>
> > > >>>> > > >>>>>>> Let me know if you have a PR for a Flink update 
> > > >>>> > > >>>>>>> :)
> > > >>>> > > >>>>>>>
> > > >>>> > > >>>>>>> On Thu, Jun 8, 2023 at 5:52 PM Galen Warren via 
> > > >>>> > > >>>>>>> user < user@flink.apache.org> wrote:
> > > >>>> > > >>>>>>>
> > > >>>> > > >>>>>>>> Thanks Martijn.
> > > >>>> > > >>>>>>>>
> > > >>>> > > >>>>>>>> Personally, I'm already using a local fork of
> Statefun
> > > >>>> that is
> > > >>>> > > >>>>>>>> compatible with Flink 1.16.x, so I wouldn't have 
> > > >>>> > > >>>>>>>> any
> > need
> > > >>>> for a
> > > >>>> > > released
> > > >>>> > > >>>>>>>> version compatible with 1.15.x. I'd be happy to 
> > > >>>> > > >>>>>>>> do
> the
> > PRs
> > > >>>> to
> > > >>>> > > modify
> > > >>>> > > >>>>>>>> Statefun to work with new versions of Flink as 
> > > >>>> > > >>>>>>>> they
> > come
> > > >>>> along.
> > > >>>> > > >>>>>>>>
> > > >>>> > > >>>>>>>> As for testing, Statefun does have unit tests 
> > > >>>> > > >>>>>>>> and
> > Gordon
> > > >>>> also
> > > >>>> > sent
> > > >>>> > > >>>>>>>> me instructions a while back for how to do some
> > additional
> > > >>>> smoke
> > > >>>> > > tests
> > > >>>> > > >>>>>>>> which are pretty straightforward. Perhaps he 
> > > >>>> > > >>>>>>>> could
> > weigh
> > > >>>> in on
> > > >>>> > > whether the
> > > >>>> > > >>>>>>>> combination of automated tests plus those smoke 
> > > >>>> > > >>>>>>>> tests
> > > >>>> should be
> > > >>>> > > sufficient
> > > >>>> > > >>>>>>>> for testing with new Flink versions (I believe 
> > > >>>> > > >>>>>>>> the
> > answer
> > > >>>> is
> > > >>>> > yes).
> > > >>>> > > >>>>>>>>
> > > >>>> > > >>>>>>>> -- Galen
> > > >>>> > > >>>>>>>>
> > > >>>> > > >>>>>>>>
> > > >>>> > > >>>>>>>>
> > > >>>> > > >>>>>>>> On Thu, Jun 8, 2023 at 8:01 AM Martijn Visser < 
> > > >>>> > > >>>>>>>> martijnvisser@apache.org> wrote:
> > > >>>> > > >>>>>>>>
> > > >>>> > > >>>>>>>>> Hi all,
> > > >>>> > > >>>>>>>>>
> > > >>>> > > >>>>>>>>> Apologies for the late reply.
> > > >>>> > > >>>>>>>>>
> > > >>>> > > >>>>>>>>> I'm willing to help out with merging requests 
> > > >>>> > > >>>>>>>>> in
> > Statefun
> > > >>>> to
> > > >>>> > keep
> > > >>>> > > >>>>>>>>> them
> > > >>>> > > >>>>>>>>> compatible with new Flink releases and create 
> > > >>>> > > >>>>>>>>> new
> > > >>>> releases. I
> > > >>>> > do
> > > >>>> > > >>>>>>>>> think that
> > > >>>> > > >>>>>>>>> validation of the functionality of these 
> > > >>>> > > >>>>>>>>> releases
> > depends
> > > >>>> a lot
> > > >>>> > > on
> > > >>>> > > >>>>>>>>> those
> > > >>>> > > >>>>>>>>> who do these compatibility updates, with PMC 
> > > >>>> > > >>>>>>>>> members
> > > >>>> helping
> > > >>>> > out
> > > >>>> > > >>>>>>>>> with the
> > > >>>> > > >>>>>>>>> formal process.
> > > >>>> > > >>>>>>>>>
> > > >>>> > > >>>>>>>>> > Why can't the Apache Software Foundation 
> > > >>>> > > >>>>>>>>> > allow
> > community
> > > >>>> > > members
> > > >>>> > > >>>>>>>>> to bring
> > > >>>> > > >>>>>>>>> it up to date?
> > > >>>> > > >>>>>>>>>
> > > >>>> > > >>>>>>>>> There's nothing preventing anyone from 
> > > >>>> > > >>>>>>>>> reviewing any
> > of
> > > >>>> the
> > > >>>> > > >>>>>>>>> current PRs or
> > > >>>> > > >>>>>>>>> opening new ones. However, none of them are 
> > > >>>> > > >>>>>>>>> approved
> > [1],
> > > >>>> so
> > > >>>> > > >>>>>>>>> there's also
> > > >>>> > > >>>>>>>>> nothing to merge.
> > > >>>> > > >>>>>>>>>
> > > >>>> > > >>>>>>>>> > I believe that there are people and companies 
> > > >>>> > > >>>>>>>>> > on
> > this
> > > >>>> mailing
> > > >>>> > > >>>>>>>>> list
> > > >>>> > > >>>>>>>>> interested in supporting Apache Flink Stateful
> > Functions.
> > > >>>> > > >>>>>>>>>
> > > >>>> > > >>>>>>>>> If so, then now is the time to show.
> > > >>>> > > >>>>>>>>>
> > > >>>> > > >>>>>>>>> Would there be a preference to create a release 
> > > >>>> > > >>>>>>>>> with
> > > >>>> Galen's
> > > >>>> > > merged
> > > >>>> > > >>>>>>>>> compatibility update to Flink 1.15.2, or do we 
> > > >>>> > > >>>>>>>>> want
> to
> > > >>>> skip
> > > >>>> > that
> > > >>>> > > >>>>>>>>> and go
> > > >>>> > > >>>>>>>>> straight to a newer version?
> > > >>>> > > >>>>>>>>>
> > > >>>> > > >>>>>>>>> Best regards,
> > > >>>> > > >>>>>>>>>
> > > >>>> > > >>>>>>>>> Martijn
> > > >>>> > > >>>>>>>>>
> > > >>>> > > >>>>>>>>> [1]
> > > >>>> > > >>>>>>>>>
> > > >>>> > > >>>>>>>>>
> > > >>>> > >
> > > >>>> >
> > > >>>>
> >
> https://github.com/apache/flink-statefun/pulls?q=is%3Apr+is%3Aopen+rev
> iew%3Aapproved
> > > >>>> > > >>>>>>>>>
> > > >>>> > > >>>>>>>>> On Tue, Jun 6, 2023 at 3:55 PM Marco Villalobos 
> > > >>>> > > >>>>>>>>> < mvillalobos@kineteque.com>
> > > >>>> > > >>>>>>>>> wrote:
> > > >>>> > > >>>>>>>>>
> > > >>>> > > >>>>>>>>> > Why can't the Apache Software Foundation 
> > > >>>> > > >>>>>>>>> > allow
> > community
> > > >>>> > > members
> > > >>>> > > >>>>>>>>> to bring
> > > >>>> > > >>>>>>>>> > it up to date?
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > What's the process for that?
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > I believe that there are people and companies 
> > > >>>> > > >>>>>>>>> > on
> > this
> > > >>>> mailing
> > > >>>> > > >>>>>>>>> list
> > > >>>> > > >>>>>>>>> > interested in supporting Apache Flink 
> > > >>>> > > >>>>>>>>> > Stateful
> > > >>>> Functions.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > You already had two people on this thread 
> > > >>>> > > >>>>>>>>> > express
> > > >>>> interest.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > At the very least, we could keep the library
> > versions
> > > >>>> up to
> > > >>>> > > date.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > There are only a small list of new features 
> > > >>>> > > >>>>>>>>> > that
> > might
> > > >>>> be
> > > >>>> > > >>>>>>>>> worthwhile:
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > 1. event time processing 2. state rest api
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <
> > > >>>> > > chesnay@apache.org>
> > > >>>> > > >>>>>>>>> wrote:
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > If you were to fork it *and want to redistribute
> it*
> > > >>>> then the
> > > >>>> > > >>>>>>>>> short
> > > >>>> > > >>>>>>>>> > version is that
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> >    1. you have to adhere to the Apache licensing
> > > >>>> requirements
> > > >>>> > > >>>>>>>>> >    2. you have to make it clear that your fork
> does
> > not
> > > >>>> > belong
> > > >>>> > > >>>>>>>>> to the
> > > >>>> > > >>>>>>>>> >    Apache Flink project. (Trademarks and all that)
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > Neither should be significant hurdles (there
> should
> > > >>>> also be
> > > >>>> > > >>>>>>>>> plenty of
> > > >>>> > > >>>>>>>>> > online resources regarding 1), and if you do this
> > then
> > > >>>> you
> > > >>>> > can
> > > >>>> > > >>>>>>>>> freely share
> > > >>>> > > >>>>>>>>> > your fork with others.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > I've also pinged Martijn to take a look at this
> > thread.
> > > >>>> > > >>>>>>>>> > To my knowledge the project hasn't decided
> anything
> > yet.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > On 27/05/2023 04:05, Galen Warren wrote:
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > Ok, I get it. No interest.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > If this project is being abandoned, I guess I'll
> > work
> > > >>>> with my
> > > >>>> > > >>>>>>>>> own fork. Is
> > > >>>> > > >>>>>>>>> > there anything I should consider here? Can I share
> > it
> > > >>>> with
> > > >>>> > > other
> > > >>>> > > >>>>>>>>> people who
> > > >>>> > > >>>>>>>>> > use this project?
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > On Tue, May 16, 2023 at 10:50 AM Galen Warren <
> > > >>>> > > >>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
> > > >>>> > > >>>>>>>>> > wrote:
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > Hi Martijn, since you opened this discussion
> > thread, I'm
> > > >>>> > > curious
> > > >>>> > > >>>>>>>>> what your
> > > >>>> > > >>>>>>>>> > thoughts are in light of the responses? Thanks.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <
> > > >>>> > > >>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
> > > >>>> > > >>>>>>>>> > wrote:
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > I use Apache Flink for stream processing, and
> > StateFun
> > > >>>> as a
> > > >>>> > > >>>>>>>>> hand-off
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > point for the rest of the application.
> > > >>>> > > >>>>>>>>> > It serves well as a bridge between a Flink
> Streaming
> > > >>>> job and
> > > >>>> > > >>>>>>>>> > micro-services.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > This is essentially how I use it as well, and I
> > would
> > > >>>> also be
> > > >>>> > > >>>>>>>>> sad to see
> > > >>>> > > >>>>>>>>> > it sunsetted. It works well; I don't know that
> > there is
> > > >>>> a lot
> > > >>>> > > of
> > > >>>> > > >>>>>>>>> new
> > > >>>> > > >>>>>>>>> > development required, but if there are no new
> > Statefun
> > > >>>> > > releases,
> > > >>>> > > >>>>>>>>> then
> > > >>>> > > >>>>>>>>> > Statefun can only be used with older Flink
> versions.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos
> <
> > > >>>> > > >>>>>>>>> mvillalobos@kineteque.com> wrote:
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > I am currently using Stateful Functions in my
> > > >>>> application.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > I use Apache Flink for stream processing, and
> > StateFun
> > > >>>> as a
> > > >>>> > > >>>>>>>>> hand-off
> > > >>>> > > >>>>>>>>> > point for the rest of the application.
> > > >>>> > > >>>>>>>>> > It serves well as a bridge between a Flink
> Streaming
> > > >>>> job and
> > > >>>> > > >>>>>>>>> > micro-services.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > I would be disappointed if StateFun was sunsetted.
> > Its
> > > >>>> a
> > > >>>> > good
> > > >>>> > > >>>>>>>>> idea.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > If there is anything I can do to help, as a
> > contributor
> > > >>>> > > perhaps,
> > > >>>> > > >>>>>>>>> please
> > > >>>> > > >>>>>>>>> > let me know.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <
> > > >>>> > > >>>>>>>>> martijnvisser@apache.org> <martijnvisser@apache.org
> >
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > wrote:
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > Hi everyone,
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > I want to open a discussion on the status of the
> > > >>>> Statefun
> > > >>>> > > >>>>>>>>> Project [1]
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > in Apache Flink. As you might have noticed, there
> > > >>>> hasn't been
> > > >>>> > > >>>>>>>>> much
> > > >>>> > > >>>>>>>>> > development over the past months in the Statefun
> > > >>>> repository
> > > >>>> > > [2].
> > > >>>> > > >>>>>>>>> There is
> > > >>>> > > >>>>>>>>> > currently a lack of active contributors and
> > committers
> > > >>>> who
> > > >>>> > are
> > > >>>> > > >>>>>>>>> able to help
> > > >>>> > > >>>>>>>>> > with the maintenance of the project.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > In order to improve the situation, we need to
> solve
> > the
> > > >>>> lack
> > > >>>> > of
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > committers and the lack of contributors.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > On the lack of committers:
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > 1. Ideally, there are some of the current Flink
> > > >>>> committers
> > > >>>> > who
> > > >>>> > > >>>>>>>>> have
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > the bandwidth and can help with reviewing PRs and
> > > >>>> merging
> > > >>>> > them.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > 2. If that's not an option, it could be a
> > consideration
> > > >>>> that
> > > >>>> > > >>>>>>>>> current
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > committers only approve and review PRs, that are
> > > >>>> approved by
> > > >>>> > > >>>>>>>>> those who are
> > > >>>> > > >>>>>>>>> > willing to contribute to Statefun and if the CI
> > passes
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > On the lack of contributors:
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > 3. Next to having this discussion on the Dev and
> > User
> > > >>>> mailing
> > > >>>> > > >>>>>>>>> list, we
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > can also create a blog with a call for new
> > contributors
> > > >>>> on
> > > >>>> > the
> > > >>>> > > >>>>>>>>> Flink
> > > >>>> > > >>>>>>>>> > project website, send out some tweets on the
> Flink /
> > > >>>> Statefun
> > > >>>> > > >>>>>>>>> twitter
> > > >>>> > > >>>>>>>>> > accounts, post messages on Slack etc. In that
> > message,
> > > >>>> we
> > > >>>> > would
> > > >>>> > > >>>>>>>>> inform how
> > > >>>> > > >>>>>>>>> > those that are interested in contributing can
> start
> > and
> > > >>>> where
> > > >>>> > > >>>>>>>>> they could
> > > >>>> > > >>>>>>>>> > reach out for more information.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > There's also option 4. where a group of interested
> > > >>>> people
> > > >>>> > would
> > > >>>> > > >>>>>>>>> split
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > Statefun from the Flink project and make it a
> > separate
> > > >>>> top
> > > >>>> > > level
> > > >>>> > > >>>>>>>>> project
> > > >>>> > > >>>>>>>>> > under the Apache Flink umbrella (similar as
> > recently has
> > > >>>> > > >>>>>>>>> happened with
> > > >>>> > > >>>>>>>>> > Flink Table Store, which has become Apache
> Paimon).
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > If we see no improvements in the coming period, we
> > > >>>> should
> > > >>>> > > >>>>>>>>> consider
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > sunsetting Statefun and communicate that clearly
> to
> > the
> > > >>>> > users.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > I'm looking forward to your thoughts.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > Best regards,
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > Martijn
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > [1]
> > > >>>> > > >>>>>>>>>
> > > >>>> https://nightlies.apache.org/flink/flink-statefun-docs-master/
> > > >>>> > <
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> >
> > > >>>> > https://nightlies.apache.org/flink/flink-statefun-docs-master/
> > > >>>> > > >
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > [2] https://github.com/apache/flink-statefun <
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > https://github.com/apache/flink-statefun>
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>>
> > > >>>> > > >>>>>>>>
> > > >>>> > >
> > > >>>> >
> > > >>>>
> > > >>>
> >
>

Re: [DISCUSS] Status of Statefun Project

[Galen Warren via dev <de...@flink.apache.org>]

I had been using it - and contributed some code - but have given up on it
and moved on to other solutions since it's clearly not being supported.

On Wed, Jan 28, 2026, 6:30 AM Martijn Visser <ma...@apache.org>
wrote:

> I forgot to include the link to this discussion thread for those that don't
> have the history, that can be found at
> https://lists.apache.org/thread/7cr2bgt91ppk6pz8o0nfbd10gs63nz6t
>
> On Wed, Jan 28, 2026 at 12:28 PM Martijn Visser <ma...@apache.org>
> wrote:
>
> > Hi everyone,
> >
> > I'm bringing back this very old thread. It's been a while since we
> > last discussed this, so I wanted to follow up on where things stand
> > with Stateful Functions.
> >
> >  Since the 3.3.0 release in September 2023, there has been:
> >  - Zero commits to the repository
> >  - No new releases (now 4+ Flink versions behind: 1.16 vs 1.20/2.2)
> >  - 22 open PRs unreviewed
> >  - No sustained contributor activity
> >
> > I don't think that the current Flink committers and PMC members have
> > the bandwidth or expertise in StateFun's codebase to maintain it.
> > StateFun was always a specialized subproject maintained by a small
> > group of dedicated contributors who have since moved on to other
> > projects.
> >
> > Given this situation, I'd like to get the community's input:
> >
> > 1. Is anyone still actively using StateFun in production?
> > 2. Should we consider sunsetting the project rather than leaving it in
> > limbo?
> >
> > Thanks,
> >
> > Martijn
> >
> > Op wo 23 aug 2023 om 08:48 schreef Filip Karnicki <
> > filip.karnicki@gmail.com>:
> > >
> > > Hi Gordon
> > >
> > > Any chance we could get this reviewed and released to the central repo?
> > > We're currently forced to use a Flink version that has a nasty bug
> > causing
> > > an operational nightmare
> > >
> > > Many thanks
> > > Fil
> > >
> > > On Sat, 19 Aug 2023 at 01:38, Galen Warren via user <
> > user@flink.apache.org>
> > > wrote:
> > >
> > > > Gotcha, makes sense as to the original division.
> > > >
> > > > >> Can this be solved by simply passing in the path to the artifacts
> > > >
> > > > This definitely works if we're going to be copying the artifacts on
> the
> > > > host side -- into the build context -- and then from the context into
> > the
> > > > image. It only gets tricky to have a potentially varying path to the
> > > > artifacts if we're trying to *directly *include the artifacts in the
> > > > Docker context -- then we have a situation where the Docker context
> > must
> > > > contain both the artifacts and playground files, with (potentially)
> > > > different root locations.
> > > >
> > > > Maybe the simplest thing to do here is just to leave the playground
> > as-is
> > > > and then copy the artifacts into the Docker context manually, prior
> to
> > > > building the playground images. I'm fine with that. It will mean that
> > each
> > > > Statefun release will require two PRs and two sets of build/publish
> > steps
> > > > instead of one, but if everyone else is fine with that I am, too.
> > Unless
> > > > anyone objects, I'll go ahead and queue up a PR for the playground
> that
> > > > makes these changes.
> > > >
> > > > Also, I should mention -- in case it's not clear -- that I have
> already
> > > > built and run the playground examples with the code from the PR and
> > > > everything worked. So that PR is ready to move forward with review,
> > etc.,
> > > > at this point.
> > > >
> > > > Thanks.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On Fri, Aug 18, 2023 at 4:16 PM Tzu-Li (Gordon) Tai <
> > tzulitai@apache.org>
> > > > wrote:
> > > >
> > > >> Hi Galen,
> > > >>
> > > >> The original intent of having a separate repo for the playground
> repo,
> > > >> was that StateFun users can just go to that and start running simple
> > > >> examples without any other distractions from the core code. I
> > personally
> > > >> don't have a strong preference here and can understand how it would
> > make
> > > >> the workflow more streamlined, but just FYI on the reasoning why are
> > > >> separate in the first place.
> > > >>
> > > >> re: paths for locating StateFun artifacts.
> > > >> Can this be solved by simply passing in the path to the artifacts?
> As
> > > >> well as the image tag for the locally build base StateFun image.
> They
> > could
> > > >> probably be environment variables.
> > > >>
> > > >> Cheers,
> > > >> Gordon
> > > >>
> > > >> On Fri, Aug 18, 2023 at 12:13 PM Galen Warren via user <
> > > >> user@flink.apache.org> wrote:
> > > >>
> > > >>> Yes, exactly! And in addition to the base Statefun jars and the jar
> > for
> > > >>> the Java SDK, it does an equivalent copy/register operation for
> each
> > of the
> > > >>> other SDK libraries (Go, Python, Javascript) so that those
> libraries
> > are
> > > >>> also available when building the playground examples.
> > > >>>
> > > >>> One more question: In order to copy the various build artifacts
> into
> > the
> > > >>> Docker containers, those artifacts need to be part of the Docker
> > context.
> > > >>> With the playground being a separate project, that's slightly
> tricky
> > to do,
> > > >>> as there is no guarantee (other than convention) about the relative
> > paths
> > > >>> of *flink-statefun* and* flink-statefun-playground *in someone's
> > local
> > > >>> filesystem. The way I've set this up locally is to copy the
> > playground into
> > > >>> the* flink-statefun* project -- i.e. to *flink-statefun*/playground
> > --
> > > >>> such that I can set the Docker context to the root folder of
> > > >>> *flink-statefun* and then have access to any local code and/or
> build
> > > >>> artifacts.
> > > >>>
> > > >>> I'm wondering if there might be any appetite for making that move
> > > >>> permanent, i.e. moving the playground to
> *flink-statefun*/playground
> > > >>> and deprecating the standalone playground project. In addition to
> > making
> > > >>> the problem of building with unreleased artifacts a bit simpler to
> > solve,
> > > >>> it would also simplify the process of releasing a new Statefun
> > version,
> > > >>> since the entire process could be handled with a single PR and
> > associated
> > > >>> build/deploy tasks. In other words, a single PR could both update
> and
> > > >>> deploy the Statefun package and the playground code and images.
> > > >>>
> > > >>> As it stands, at least two PRs would be required for each Statefun
> > > >>> version update -- one for *flink-statefun* and one for
> > > >>> *flink-statefun-playground*.
> > > >>>
> > > >>> Anyway, just an idea. Maybe there's an important reason for these
> > > >>> projects to remain separate. If we do want to keep the playground
> > project
> > > >>> where it is, I could solve the copying problem by requiring the two
> > > >>> projects to be siblings in the file system and by pre-copying the
> > local
> > > >>> build artifacts into a location accessible by the existing Docker
> > contexts.
> > > >>> This would still leave us with the need to have two PRs and
> releases
> > > >>> instead of one, though.
> > > >>>
> > > >>> Thanks for your help!
> > > >>>
> > > >>>
> > > >>> On Fri, Aug 18, 2023 at 2:45 PM Tzu-Li (Gordon) Tai <
> > tzulitai@apache.org>
> > > >>> wrote:
> > > >>>
> > > >>>> Hi Galen,
> > > >>>>
> > > >>>> > locally built code is copied into the build containers
> > > >>>> so that it can be accessed during the build.
> > > >>>>
> > > >>>> That's exactly what we had been doing for release testing, yes.
> > Sorry I
> > > >>>> missed that detail in my previous response.
> > > >>>>
> > > >>>> And yes, that sounds like a reasonable approach. If I understand
> you
> > > >>>> correctly, the workflow would become this:
> > > >>>>
> > > >>>>    1. Build the StateFun repo locally to install the snapshot
> > artifact
> > > >>>>    jars + have a local base StateFun image.
> > > >>>>    2. Run the playground in "local" mode, so that it uses the
> local
> > base
> > > >>>>    StateFun image + builds the playground code using copied
> artifact
> > > >>>> jars
> > > >>>>    (instead of pulling from Maven).
> > > >>>>
> > > >>>> That looks good to me!
> > > >>>>
> > > >>>> Thanks,
> > > >>>> Gordon
> > > >>>>
> > > >>>> On Fri, Aug 18, 2023 at 11:33 AM Galen Warren
> > > >>>> <ga...@cvillewarrens.com.invalid> wrote:
> > > >>>>
> > > >>>> > Thanks.
> > > >>>> >
> > > >>>> > If you were to build a local image, as you suggest, how do you
> > access
> > > >>>> that
> > > >>>> > image when building the playground images? All the compilation
> of
> > > >>>> > playground code happens inside containers, so local images on
> the
> > host
> > > >>>> > aren't available in those containers. Unless I'm missing
> > something?
> > > >>>> >
> > > >>>> > I've slightly reworked things such that the playground images
> can
> > be
> > > >>>> run in
> > > >>>> > one of two modes -- the default mode, which works like before,
> > and a
> > > >>>> > "local" mode where locally built code is copied into the build
> > > >>>> containers
> > > >>>> > so that it can be accessed during the build. It works fine, you
> > just
> > > >>>> have
> > > >>>> > to define a couple of environment variables when running
> > > >>>> docker-compose to
> > > >>>> > specify default vs. local mode and what versions of Flink and
> > Statefun
> > > >>>> > should be referenced, and then you can build a run the local
> > examples
> > > >>>> > without any additional steps. Does that sound like a reasonable
> > > >>>> approach?
> > > >>>> >
> > > >>>> >
> > > >>>> > On Fri, Aug 18, 2023 at 2:17 PM Tzu-Li (Gordon) Tai <
> > > >>>> tzulitai@apache.org>
> > > >>>> > wrote:
> > > >>>> >
> > > >>>> > > Hi Galen,
> > > >>>> > >
> > > >>>> > > > Gordon, is there a trick to running the sample code in
> > > >>>> > > flink-statefun-playground against yet-unreleased code that I'm
> > > >>>> missing?
> > > >>>> > >
> > > >>>> > > You'd have to locally build an image from the release branch,
> > with a
> > > >>>> > > temporary image version tag. Then, in the
> > flink-statefun-playground,
> > > >>>> > change
> > > >>>> > > the image versions in the docker-compose files to use that
> > locally
> > > >>>> built
> > > >>>> > > image. IIRC, that's what we have been doing in the past.
> > > >>>> Admittedly, it's
> > > >>>> > > pretty manual - I don't think the CI manages this workflow.
> > > >>>> > >
> > > >>>> > > Thanks,
> > > >>>> > > Gordon
> > > >>>> > >
> > > >>>> > > On Mon, Aug 14, 2023 at 10:42 AM Galen Warren <
> > > >>>> galen@cvillewarrens.com>
> > > >>>> > > wrote:
> > > >>>> > >
> > > >>>> > > > I created a pull request for this: [FLINK-31619] Upgrade
> > Stateful
> > > >>>> > > > Functions to Flink 1.16.1 by galenwarren · Pull Request
> #331 ·
> > > >>>> > > > apache/flink-statefun (github.com)
> > > >>>> > > > <https://github.com/apache/flink-statefun/pull/331>.
> > > >>>> > > >
> > > >>>> > > > JIRA is here: [FLINK-31619] Upgrade Stateful Functions to
> > Flink
> > > >>>> 1.16.1
> > > >>>> > -
> > > >>>> > > > ASF JIRA (apache.org)
> > > >>>> > > > <
> https://issues.apache.org/jira/browse/FLINK-31619?filter=-1
> > >.
> > > >>>> > > >
> > > >>>> > > > Statefun references 1.16.2, despite the title -- that
> version
> > has
> > > >>>> come
> > > >>>> > > out
> > > >>>> > > > since the issue was created.
> > > >>>> > > >
> > > >>>> > > > I figured out how to run all the playground tests locally,
> > but it
> > > >>>> took
> > > >>>> > a
> > > >>>> > > > bit of reworking of the playground setup with respect to
> > Docker;
> > > >>>> > > > specifically, the Docker contexts used to build the example
> > > >>>> functions
> > > >>>> > > > needed to be broadened (i.e. moved up the tree) so that, if
> > > >>>> needed,
> > > >>>> > local
> > > >>>> > > > artifacts/code can be accessed from within the containers at
> > build
> > > >>>> > time.
> > > >>>> > > > Then I made the Docker compose.yml configurable through
> > > >>>> environment
> > > >>>> > > > variables to allow for them to run in either the original
> > manner
> > > >>>> --
> > > >>>> > i.e.
> > > >>>> > > > pulling artifacts from public repos -- or in a "local" mode,
> > where
> > > >>>> > > > artifacts are pulled from local builds.
> > > >>>> > > >
> > > >>>> > > > This process is a cleaner if the playground is a subfolder
> of
> > the
> > > >>>> > > > flink-statefun project rather than be its own repository
> > > >>>> > > > (flink-statefun-playground), because then all the relative
> > paths
> > > >>>> > between
> > > >>>> > > > the playground files and the build artifacts are fixed. So,
> > I'd
> > > >>>> like to
> > > >>>> > > > propose to move the playground files, modified as described
> > > >>>> above, to
> > > >>>> > > > flink-statefun/playground and retire
> > flink-statefun-playground. I
> > > >>>> can
> > > >>>> > > > submit separate PR s those changes if everyone is on board.
> > > >>>> > > >
> > > >>>> > > > Also, should I plan to do the same upgrade to handle Flink
> > > >>>> 1.17.x? It
> > > >>>> > > > should be easy to do, especially while the 1.16.x upgrade is
> > > >>>> fresh on
> > > >>>> > my
> > > >>>> > > > mind.
> > > >>>> > > >
> > > >>>> > > > Thanks.
> > > >>>> > > >
> > > >>>> > > >
> > > >>>> > > > On Fri, Aug 11, 2023 at 6:40 PM Galen Warren <
> > > >>>> galen@cvillewarrens.com>
> > > >>>> > > > wrote:
> > > >>>> > > >
> > > >>>> > > >> I'm done with the code to make Statefun compatible with
> Flink
> > > >>>> 1.16,
> > > >>>> > and
> > > >>>> > > >> all the tests (including e2e succeed). The required changes
> > were
> > > >>>> > pretty
> > > >>>> > > >> minimal.
> > > >>>> > > >>
> > > >>>> > > >> I'm running into a bit of a chicken/egg problem executing
> the
> > > >>>> tests in
> > > >>>> > > >> flink-statefun-playground
> > > >>>> > > >> <https://github.com/apache/flink-statefun-playground>,
> > though.
> > > >>>> That
> > > >>>> > > >> project seems to assume that all the various Statefun
> > artifacts
> > > >>>> are
> > > >>>> > > built
> > > >>>> > > >> and deployed to the various public repositories already.
> I've
> > > >>>> looked
> > > >>>> > > into
> > > >>>> > > >> trying to redirect references to local artifacts; however,
> > > >>>> that's also
> > > >>>> > > >> tricky since all the building occurs in Docker containers.
> > > >>>> > > >>
> > > >>>> > > >> Gordon, is there a trick to running the sample code in
> > > >>>> > > >> flink-statefun-playground against yet-unreleased code that
> > I'm
> > > >>>> > missing?
> > > >>>> > > >>
> > > >>>> > > >> Thanks.
> > > >>>> > > >>
> > > >>>> > > >> On Sat, Jun 24, 2023 at 12:40 PM Galen Warren <
> > > >>>> > galen@cvillewarrens.com>
> > > >>>> > > >> wrote:
> > > >>>> > > >>
> > > >>>> > > >>> Great -- thanks!
> > > >>>> > > >>>
> > > >>>> > > >>> I'm going to be out of town for about a week but I'll
> take a
> > > >>>> look at
> > > >>>> > > >>> this when I'm back.
> > > >>>> > > >>>
> > > >>>> > > >>> On Tue, Jun 20, 2023 at 8:46 AM Martijn Visser <
> > > >>>> mvisser@confluent.io
> > > >>>> > >
> > > >>>> > > >>> wrote:
> > > >>>> > > >>>
> > > >>>> > > >>>> Hi Galen,
> > > >>>> > > >>>>
> > > >>>> > > >>>> Yes, I'll be more than happy to help with Statefun
> > releases.
> > > >>>> > > >>>>
> > > >>>> > > >>>> Best regards,
> > > >>>> > > >>>>
> > > >>>> > > >>>> Martijn
> > > >>>> > > >>>>
> > > >>>> > > >>>> On Tue, Jun 20, 2023 at 2:21 PM Galen Warren <
> > > >>>> > galen@cvillewarrens.com
> > > >>>> > > >
> > > >>>> > > >>>> wrote:
> > > >>>> > > >>>>
> > > >>>> > > >>>>> Thanks.
> > > >>>> > > >>>>>
> > > >>>> > > >>>>> Martijn, to answer your question, I'd need to do a small
> > > >>>> amount of
> > > >>>> > > >>>>> work to get a PR ready, but not much. Happy to do it if
> > we're
> > > >>>> > > deciding to
> > > >>>> > > >>>>> restart Statefun releases -- are we?
> > > >>>> > > >>>>>
> > > >>>> > > >>>>> -- Galen
> > > >>>> > > >>>>>
> > > >>>> > > >>>>> On Sat, Jun 17, 2023 at 9:47 AM Tzu-Li (Gordon) Tai <
> > > >>>> > > >>>>> tzulitai@apache.org> wrote:
> > > >>>> > > >>>>>
> > > >>>> > > >>>>>> > Perhaps he could weigh in on whether the combination
> of
> > > >>>> > automated
> > > >>>> > > >>>>>> tests plus those smoke tests should be sufficient for
> > > >>>> testing with
> > > >>>> > > new
> > > >>>> > > >>>>>> Flink versions
> > > >>>> > > >>>>>>
> > > >>>> > > >>>>>> What we usually did at the bare minimum for new
> StateFun
> > > >>>> releases
> > > >>>> > > was
> > > >>>> > > >>>>>> the following:
> > > >>>> > > >>>>>>
> > > >>>> > > >>>>>>    1. Build tests (including the smoke tests in the e2e
> > > >>>> module,
> > > >>>> > > >>>>>>    which covers important tests like exactly-once
> > > >>>> verification)
> > > >>>> > > >>>>>>    2. Updating the flink-statefun-playground repo and
> > > >>>> manually
> > > >>>> > > >>>>>>    running all language examples there.
> > > >>>> > > >>>>>>
> > > >>>> > > >>>>>> If upgrading Flink versions was the only change in the
> > > >>>> release,
> > > >>>> > I'd
> > > >>>> > > >>>>>> probably say that this is sufficient.
> > > >>>> > > >>>>>>
> > > >>>> > > >>>>>> Best,
> > > >>>> > > >>>>>> Gordon
> > > >>>> > > >>>>>>
> > > >>>> > > >>>>>> On Thu, Jun 15, 2023 at 5:25 AM Martijn Visser <
> > > >>>> > > >>>>>> martijnvisser@apache.org> wrote:
> > > >>>> > > >>>>>>
> > > >>>> > > >>>>>>> Let me know if you have a PR for a Flink update :)
> > > >>>> > > >>>>>>>
> > > >>>> > > >>>>>>> On Thu, Jun 8, 2023 at 5:52 PM Galen Warren via user <
> > > >>>> > > >>>>>>> user@flink.apache.org> wrote:
> > > >>>> > > >>>>>>>
> > > >>>> > > >>>>>>>> Thanks Martijn.
> > > >>>> > > >>>>>>>>
> > > >>>> > > >>>>>>>> Personally, I'm already using a local fork of
> Statefun
> > > >>>> that is
> > > >>>> > > >>>>>>>> compatible with Flink 1.16.x, so I wouldn't have any
> > need
> > > >>>> for a
> > > >>>> > > released
> > > >>>> > > >>>>>>>> version compatible with 1.15.x. I'd be happy to do
> the
> > PRs
> > > >>>> to
> > > >>>> > > modify
> > > >>>> > > >>>>>>>> Statefun to work with new versions of Flink as they
> > come
> > > >>>> along.
> > > >>>> > > >>>>>>>>
> > > >>>> > > >>>>>>>> As for testing, Statefun does have unit tests and
> > Gordon
> > > >>>> also
> > > >>>> > sent
> > > >>>> > > >>>>>>>> me instructions a while back for how to do some
> > additional
> > > >>>> smoke
> > > >>>> > > tests
> > > >>>> > > >>>>>>>> which are pretty straightforward. Perhaps he could
> > weigh
> > > >>>> in on
> > > >>>> > > whether the
> > > >>>> > > >>>>>>>> combination of automated tests plus those smoke tests
> > > >>>> should be
> > > >>>> > > sufficient
> > > >>>> > > >>>>>>>> for testing with new Flink versions (I believe the
> > answer
> > > >>>> is
> > > >>>> > yes).
> > > >>>> > > >>>>>>>>
> > > >>>> > > >>>>>>>> -- Galen
> > > >>>> > > >>>>>>>>
> > > >>>> > > >>>>>>>>
> > > >>>> > > >>>>>>>>
> > > >>>> > > >>>>>>>> On Thu, Jun 8, 2023 at 8:01 AM Martijn Visser <
> > > >>>> > > >>>>>>>> martijnvisser@apache.org> wrote:
> > > >>>> > > >>>>>>>>
> > > >>>> > > >>>>>>>>> Hi all,
> > > >>>> > > >>>>>>>>>
> > > >>>> > > >>>>>>>>> Apologies for the late reply.
> > > >>>> > > >>>>>>>>>
> > > >>>> > > >>>>>>>>> I'm willing to help out with merging requests in
> > Statefun
> > > >>>> to
> > > >>>> > keep
> > > >>>> > > >>>>>>>>> them
> > > >>>> > > >>>>>>>>> compatible with new Flink releases and create new
> > > >>>> releases. I
> > > >>>> > do
> > > >>>> > > >>>>>>>>> think that
> > > >>>> > > >>>>>>>>> validation of the functionality of these releases
> > depends
> > > >>>> a lot
> > > >>>> > > on
> > > >>>> > > >>>>>>>>> those
> > > >>>> > > >>>>>>>>> who do these compatibility updates, with PMC members
> > > >>>> helping
> > > >>>> > out
> > > >>>> > > >>>>>>>>> with the
> > > >>>> > > >>>>>>>>> formal process.
> > > >>>> > > >>>>>>>>>
> > > >>>> > > >>>>>>>>> > Why can't the Apache Software Foundation allow
> > community
> > > >>>> > > members
> > > >>>> > > >>>>>>>>> to bring
> > > >>>> > > >>>>>>>>> it up to date?
> > > >>>> > > >>>>>>>>>
> > > >>>> > > >>>>>>>>> There's nothing preventing anyone from reviewing any
> > of
> > > >>>> the
> > > >>>> > > >>>>>>>>> current PRs or
> > > >>>> > > >>>>>>>>> opening new ones. However, none of them are approved
> > [1],
> > > >>>> so
> > > >>>> > > >>>>>>>>> there's also
> > > >>>> > > >>>>>>>>> nothing to merge.
> > > >>>> > > >>>>>>>>>
> > > >>>> > > >>>>>>>>> > I believe that there are people and companies on
> > this
> > > >>>> mailing
> > > >>>> > > >>>>>>>>> list
> > > >>>> > > >>>>>>>>> interested in supporting Apache Flink Stateful
> > Functions.
> > > >>>> > > >>>>>>>>>
> > > >>>> > > >>>>>>>>> If so, then now is the time to show.
> > > >>>> > > >>>>>>>>>
> > > >>>> > > >>>>>>>>> Would there be a preference to create a release with
> > > >>>> Galen's
> > > >>>> > > merged
> > > >>>> > > >>>>>>>>> compatibility update to Flink 1.15.2, or do we want
> to
> > > >>>> skip
> > > >>>> > that
> > > >>>> > > >>>>>>>>> and go
> > > >>>> > > >>>>>>>>> straight to a newer version?
> > > >>>> > > >>>>>>>>>
> > > >>>> > > >>>>>>>>> Best regards,
> > > >>>> > > >>>>>>>>>
> > > >>>> > > >>>>>>>>> Martijn
> > > >>>> > > >>>>>>>>>
> > > >>>> > > >>>>>>>>> [1]
> > > >>>> > > >>>>>>>>>
> > > >>>> > > >>>>>>>>>
> > > >>>> > >
> > > >>>> >
> > > >>>>
> >
> https://github.com/apache/flink-statefun/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved
> > > >>>> > > >>>>>>>>>
> > > >>>> > > >>>>>>>>> On Tue, Jun 6, 2023 at 3:55 PM Marco Villalobos <
> > > >>>> > > >>>>>>>>> mvillalobos@kineteque.com>
> > > >>>> > > >>>>>>>>> wrote:
> > > >>>> > > >>>>>>>>>
> > > >>>> > > >>>>>>>>> > Why can't the Apache Software Foundation allow
> > community
> > > >>>> > > members
> > > >>>> > > >>>>>>>>> to bring
> > > >>>> > > >>>>>>>>> > it up to date?
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > What's the process for that?
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > I believe that there are people and companies on
> > this
> > > >>>> mailing
> > > >>>> > > >>>>>>>>> list
> > > >>>> > > >>>>>>>>> > interested in supporting Apache Flink Stateful
> > > >>>> Functions.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > You already had two people on this thread express
> > > >>>> interest.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > At the very least, we could keep the library
> > versions
> > > >>>> up to
> > > >>>> > > date.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > There are only a small list of new features that
> > might
> > > >>>> be
> > > >>>> > > >>>>>>>>> worthwhile:
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > 1. event time processing
> > > >>>> > > >>>>>>>>> > 2. state rest api
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <
> > > >>>> > > chesnay@apache.org>
> > > >>>> > > >>>>>>>>> wrote:
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > If you were to fork it *and want to redistribute
> it*
> > > >>>> then the
> > > >>>> > > >>>>>>>>> short
> > > >>>> > > >>>>>>>>> > version is that
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> >    1. you have to adhere to the Apache licensing
> > > >>>> requirements
> > > >>>> > > >>>>>>>>> >    2. you have to make it clear that your fork
> does
> > not
> > > >>>> > belong
> > > >>>> > > >>>>>>>>> to the
> > > >>>> > > >>>>>>>>> >    Apache Flink project. (Trademarks and all that)
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > Neither should be significant hurdles (there
> should
> > > >>>> also be
> > > >>>> > > >>>>>>>>> plenty of
> > > >>>> > > >>>>>>>>> > online resources regarding 1), and if you do this
> > then
> > > >>>> you
> > > >>>> > can
> > > >>>> > > >>>>>>>>> freely share
> > > >>>> > > >>>>>>>>> > your fork with others.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > I've also pinged Martijn to take a look at this
> > thread.
> > > >>>> > > >>>>>>>>> > To my knowledge the project hasn't decided
> anything
> > yet.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > On 27/05/2023 04:05, Galen Warren wrote:
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > Ok, I get it. No interest.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > If this project is being abandoned, I guess I'll
> > work
> > > >>>> with my
> > > >>>> > > >>>>>>>>> own fork. Is
> > > >>>> > > >>>>>>>>> > there anything I should consider here? Can I share
> > it
> > > >>>> with
> > > >>>> > > other
> > > >>>> > > >>>>>>>>> people who
> > > >>>> > > >>>>>>>>> > use this project?
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > On Tue, May 16, 2023 at 10:50 AM Galen Warren <
> > > >>>> > > >>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
> > > >>>> > > >>>>>>>>> > wrote:
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > Hi Martijn, since you opened this discussion
> > thread, I'm
> > > >>>> > > curious
> > > >>>> > > >>>>>>>>> what your
> > > >>>> > > >>>>>>>>> > thoughts are in light of the responses? Thanks.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <
> > > >>>> > > >>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
> > > >>>> > > >>>>>>>>> > wrote:
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > I use Apache Flink for stream processing, and
> > StateFun
> > > >>>> as a
> > > >>>> > > >>>>>>>>> hand-off
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > point for the rest of the application.
> > > >>>> > > >>>>>>>>> > It serves well as a bridge between a Flink
> Streaming
> > > >>>> job and
> > > >>>> > > >>>>>>>>> > micro-services.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > This is essentially how I use it as well, and I
> > would
> > > >>>> also be
> > > >>>> > > >>>>>>>>> sad to see
> > > >>>> > > >>>>>>>>> > it sunsetted. It works well; I don't know that
> > there is
> > > >>>> a lot
> > > >>>> > > of
> > > >>>> > > >>>>>>>>> new
> > > >>>> > > >>>>>>>>> > development required, but if there are no new
> > Statefun
> > > >>>> > > releases,
> > > >>>> > > >>>>>>>>> then
> > > >>>> > > >>>>>>>>> > Statefun can only be used with older Flink
> versions.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos
> <
> > > >>>> > > >>>>>>>>> mvillalobos@kineteque.com> wrote:
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > I am currently using Stateful Functions in my
> > > >>>> application.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > I use Apache Flink for stream processing, and
> > StateFun
> > > >>>> as a
> > > >>>> > > >>>>>>>>> hand-off
> > > >>>> > > >>>>>>>>> > point for the rest of the application.
> > > >>>> > > >>>>>>>>> > It serves well as a bridge between a Flink
> Streaming
> > > >>>> job and
> > > >>>> > > >>>>>>>>> > micro-services.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > I would be disappointed if StateFun was sunsetted.
> > Its
> > > >>>> a
> > > >>>> > good
> > > >>>> > > >>>>>>>>> idea.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > If there is anything I can do to help, as a
> > contributor
> > > >>>> > > perhaps,
> > > >>>> > > >>>>>>>>> please
> > > >>>> > > >>>>>>>>> > let me know.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <
> > > >>>> > > >>>>>>>>> martijnvisser@apache.org> <martijnvisser@apache.org
> >
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > wrote:
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > Hi everyone,
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > I want to open a discussion on the status of the
> > > >>>> Statefun
> > > >>>> > > >>>>>>>>> Project [1]
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > in Apache Flink. As you might have noticed, there
> > > >>>> hasn't been
> > > >>>> > > >>>>>>>>> much
> > > >>>> > > >>>>>>>>> > development over the past months in the Statefun
> > > >>>> repository
> > > >>>> > > [2].
> > > >>>> > > >>>>>>>>> There is
> > > >>>> > > >>>>>>>>> > currently a lack of active contributors and
> > committers
> > > >>>> who
> > > >>>> > are
> > > >>>> > > >>>>>>>>> able to help
> > > >>>> > > >>>>>>>>> > with the maintenance of the project.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > In order to improve the situation, we need to
> solve
> > the
> > > >>>> lack
> > > >>>> > of
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > committers and the lack of contributors.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > On the lack of committers:
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > 1. Ideally, there are some of the current Flink
> > > >>>> committers
> > > >>>> > who
> > > >>>> > > >>>>>>>>> have
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > the bandwidth and can help with reviewing PRs and
> > > >>>> merging
> > > >>>> > them.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > 2. If that's not an option, it could be a
> > consideration
> > > >>>> that
> > > >>>> > > >>>>>>>>> current
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > committers only approve and review PRs, that are
> > > >>>> approved by
> > > >>>> > > >>>>>>>>> those who are
> > > >>>> > > >>>>>>>>> > willing to contribute to Statefun and if the CI
> > passes
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > On the lack of contributors:
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > 3. Next to having this discussion on the Dev and
> > User
> > > >>>> mailing
> > > >>>> > > >>>>>>>>> list, we
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > can also create a blog with a call for new
> > contributors
> > > >>>> on
> > > >>>> > the
> > > >>>> > > >>>>>>>>> Flink
> > > >>>> > > >>>>>>>>> > project website, send out some tweets on the
> Flink /
> > > >>>> Statefun
> > > >>>> > > >>>>>>>>> twitter
> > > >>>> > > >>>>>>>>> > accounts, post messages on Slack etc. In that
> > message,
> > > >>>> we
> > > >>>> > would
> > > >>>> > > >>>>>>>>> inform how
> > > >>>> > > >>>>>>>>> > those that are interested in contributing can
> start
> > and
> > > >>>> where
> > > >>>> > > >>>>>>>>> they could
> > > >>>> > > >>>>>>>>> > reach out for more information.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > There's also option 4. where a group of interested
> > > >>>> people
> > > >>>> > would
> > > >>>> > > >>>>>>>>> split
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > Statefun from the Flink project and make it a
> > separate
> > > >>>> top
> > > >>>> > > level
> > > >>>> > > >>>>>>>>> project
> > > >>>> > > >>>>>>>>> > under the Apache Flink umbrella (similar as
> > recently has
> > > >>>> > > >>>>>>>>> happened with
> > > >>>> > > >>>>>>>>> > Flink Table Store, which has become Apache
> Paimon).
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > If we see no improvements in the coming period, we
> > > >>>> should
> > > >>>> > > >>>>>>>>> consider
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > sunsetting Statefun and communicate that clearly
> to
> > the
> > > >>>> > users.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > I'm looking forward to your thoughts.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > Best regards,
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > Martijn
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > [1]
> > > >>>> > > >>>>>>>>>
> > > >>>> https://nightlies.apache.org/flink/flink-statefun-docs-master/
> > > >>>> > <
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> >
> > > >>>> > https://nightlies.apache.org/flink/flink-statefun-docs-master/
> > > >>>> > > >
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > [2] https://github.com/apache/flink-statefun <
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > https://github.com/apache/flink-statefun>
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>>
> > > >>>> > > >>>>>>>>
> > > >>>> > >
> > > >>>> >
> > > >>>>
> > > >>>
> >
>

Re: [DISCUSS] Status of Statefun Project

[Galen Warren via user <us...@flink.apache.org>]

I had been using it - and contributed some code - but have given up on it
and moved on to other solutions since it's clearly not being supported.

On Wed, Jan 28, 2026, 6:30 AM Martijn Visser <ma...@apache.org>
wrote:

> I forgot to include the link to this discussion thread for those that don't
> have the history, that can be found at
> https://lists.apache.org/thread/7cr2bgt91ppk6pz8o0nfbd10gs63nz6t
>
> On Wed, Jan 28, 2026 at 12:28 PM Martijn Visser <ma...@apache.org>
> wrote:
>
> > Hi everyone,
> >
> > I'm bringing back this very old thread. It's been a while since we
> > last discussed this, so I wanted to follow up on where things stand
> > with Stateful Functions.
> >
> >  Since the 3.3.0 release in September 2023, there has been:
> >  - Zero commits to the repository
> >  - No new releases (now 4+ Flink versions behind: 1.16 vs 1.20/2.2)
> >  - 22 open PRs unreviewed
> >  - No sustained contributor activity
> >
> > I don't think that the current Flink committers and PMC members have
> > the bandwidth or expertise in StateFun's codebase to maintain it.
> > StateFun was always a specialized subproject maintained by a small
> > group of dedicated contributors who have since moved on to other
> > projects.
> >
> > Given this situation, I'd like to get the community's input:
> >
> > 1. Is anyone still actively using StateFun in production?
> > 2. Should we consider sunsetting the project rather than leaving it in
> > limbo?
> >
> > Thanks,
> >
> > Martijn
> >
> > Op wo 23 aug 2023 om 08:48 schreef Filip Karnicki <
> > filip.karnicki@gmail.com>:
> > >
> > > Hi Gordon
> > >
> > > Any chance we could get this reviewed and released to the central repo?
> > > We're currently forced to use a Flink version that has a nasty bug
> > causing
> > > an operational nightmare
> > >
> > > Many thanks
> > > Fil
> > >
> > > On Sat, 19 Aug 2023 at 01:38, Galen Warren via user <
> > user@flink.apache.org>
> > > wrote:
> > >
> > > > Gotcha, makes sense as to the original division.
> > > >
> > > > >> Can this be solved by simply passing in the path to the artifacts
> > > >
> > > > This definitely works if we're going to be copying the artifacts on
> the
> > > > host side -- into the build context -- and then from the context into
> > the
> > > > image. It only gets tricky to have a potentially varying path to the
> > > > artifacts if we're trying to *directly *include the artifacts in the
> > > > Docker context -- then we have a situation where the Docker context
> > must
> > > > contain both the artifacts and playground files, with (potentially)
> > > > different root locations.
> > > >
> > > > Maybe the simplest thing to do here is just to leave the playground
> > as-is
> > > > and then copy the artifacts into the Docker context manually, prior
> to
> > > > building the playground images. I'm fine with that. It will mean that
> > each
> > > > Statefun release will require two PRs and two sets of build/publish
> > steps
> > > > instead of one, but if everyone else is fine with that I am, too.
> > Unless
> > > > anyone objects, I'll go ahead and queue up a PR for the playground
> that
> > > > makes these changes.
> > > >
> > > > Also, I should mention -- in case it's not clear -- that I have
> already
> > > > built and run the playground examples with the code from the PR and
> > > > everything worked. So that PR is ready to move forward with review,
> > etc.,
> > > > at this point.
> > > >
> > > > Thanks.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On Fri, Aug 18, 2023 at 4:16 PM Tzu-Li (Gordon) Tai <
> > tzulitai@apache.org>
> > > > wrote:
> > > >
> > > >> Hi Galen,
> > > >>
> > > >> The original intent of having a separate repo for the playground
> repo,
> > > >> was that StateFun users can just go to that and start running simple
> > > >> examples without any other distractions from the core code. I
> > personally
> > > >> don't have a strong preference here and can understand how it would
> > make
> > > >> the workflow more streamlined, but just FYI on the reasoning why are
> > > >> separate in the first place.
> > > >>
> > > >> re: paths for locating StateFun artifacts.
> > > >> Can this be solved by simply passing in the path to the artifacts?
> As
> > > >> well as the image tag for the locally build base StateFun image.
> They
> > could
> > > >> probably be environment variables.
> > > >>
> > > >> Cheers,
> > > >> Gordon
> > > >>
> > > >> On Fri, Aug 18, 2023 at 12:13 PM Galen Warren via user <
> > > >> user@flink.apache.org> wrote:
> > > >>
> > > >>> Yes, exactly! And in addition to the base Statefun jars and the jar
> > for
> > > >>> the Java SDK, it does an equivalent copy/register operation for
> each
> > of the
> > > >>> other SDK libraries (Go, Python, Javascript) so that those
> libraries
> > are
> > > >>> also available when building the playground examples.
> > > >>>
> > > >>> One more question: In order to copy the various build artifacts
> into
> > the
> > > >>> Docker containers, those artifacts need to be part of the Docker
> > context.
> > > >>> With the playground being a separate project, that's slightly
> tricky
> > to do,
> > > >>> as there is no guarantee (other than convention) about the relative
> > paths
> > > >>> of *flink-statefun* and* flink-statefun-playground *in someone's
> > local
> > > >>> filesystem. The way I've set this up locally is to copy the
> > playground into
> > > >>> the* flink-statefun* project -- i.e. to *flink-statefun*/playground
> > --
> > > >>> such that I can set the Docker context to the root folder of
> > > >>> *flink-statefun* and then have access to any local code and/or
> build
> > > >>> artifacts.
> > > >>>
> > > >>> I'm wondering if there might be any appetite for making that move
> > > >>> permanent, i.e. moving the playground to
> *flink-statefun*/playground
> > > >>> and deprecating the standalone playground project. In addition to
> > making
> > > >>> the problem of building with unreleased artifacts a bit simpler to
> > solve,
> > > >>> it would also simplify the process of releasing a new Statefun
> > version,
> > > >>> since the entire process could be handled with a single PR and
> > associated
> > > >>> build/deploy tasks. In other words, a single PR could both update
> and
> > > >>> deploy the Statefun package and the playground code and images.
> > > >>>
> > > >>> As it stands, at least two PRs would be required for each Statefun
> > > >>> version update -- one for *flink-statefun* and one for
> > > >>> *flink-statefun-playground*.
> > > >>>
> > > >>> Anyway, just an idea. Maybe there's an important reason for these
> > > >>> projects to remain separate. If we do want to keep the playground
> > project
> > > >>> where it is, I could solve the copying problem by requiring the two
> > > >>> projects to be siblings in the file system and by pre-copying the
> > local
> > > >>> build artifacts into a location accessible by the existing Docker
> > contexts.
> > > >>> This would still leave us with the need to have two PRs and
> releases
> > > >>> instead of one, though.
> > > >>>
> > > >>> Thanks for your help!
> > > >>>
> > > >>>
> > > >>> On Fri, Aug 18, 2023 at 2:45 PM Tzu-Li (Gordon) Tai <
> > tzulitai@apache.org>
> > > >>> wrote:
> > > >>>
> > > >>>> Hi Galen,
> > > >>>>
> > > >>>> > locally built code is copied into the build containers
> > > >>>> so that it can be accessed during the build.
> > > >>>>
> > > >>>> That's exactly what we had been doing for release testing, yes.
> > Sorry I
> > > >>>> missed that detail in my previous response.
> > > >>>>
> > > >>>> And yes, that sounds like a reasonable approach. If I understand
> you
> > > >>>> correctly, the workflow would become this:
> > > >>>>
> > > >>>>    1. Build the StateFun repo locally to install the snapshot
> > artifact
> > > >>>>    jars + have a local base StateFun image.
> > > >>>>    2. Run the playground in "local" mode, so that it uses the
> local
> > base
> > > >>>>    StateFun image + builds the playground code using copied
> artifact
> > > >>>> jars
> > > >>>>    (instead of pulling from Maven).
> > > >>>>
> > > >>>> That looks good to me!
> > > >>>>
> > > >>>> Thanks,
> > > >>>> Gordon
> > > >>>>
> > > >>>> On Fri, Aug 18, 2023 at 11:33 AM Galen Warren
> > > >>>> <ga...@cvillewarrens.com.invalid> wrote:
> > > >>>>
> > > >>>> > Thanks.
> > > >>>> >
> > > >>>> > If you were to build a local image, as you suggest, how do you
> > access
> > > >>>> that
> > > >>>> > image when building the playground images? All the compilation
> of
> > > >>>> > playground code happens inside containers, so local images on
> the
> > host
> > > >>>> > aren't available in those containers. Unless I'm missing
> > something?
> > > >>>> >
> > > >>>> > I've slightly reworked things such that the playground images
> can
> > be
> > > >>>> run in
> > > >>>> > one of two modes -- the default mode, which works like before,
> > and a
> > > >>>> > "local" mode where locally built code is copied into the build
> > > >>>> containers
> > > >>>> > so that it can be accessed during the build. It works fine, you
> > just
> > > >>>> have
> > > >>>> > to define a couple of environment variables when running
> > > >>>> docker-compose to
> > > >>>> > specify default vs. local mode and what versions of Flink and
> > Statefun
> > > >>>> > should be referenced, and then you can build a run the local
> > examples
> > > >>>> > without any additional steps. Does that sound like a reasonable
> > > >>>> approach?
> > > >>>> >
> > > >>>> >
> > > >>>> > On Fri, Aug 18, 2023 at 2:17 PM Tzu-Li (Gordon) Tai <
> > > >>>> tzulitai@apache.org>
> > > >>>> > wrote:
> > > >>>> >
> > > >>>> > > Hi Galen,
> > > >>>> > >
> > > >>>> > > > Gordon, is there a trick to running the sample code in
> > > >>>> > > flink-statefun-playground against yet-unreleased code that I'm
> > > >>>> missing?
> > > >>>> > >
> > > >>>> > > You'd have to locally build an image from the release branch,
> > with a
> > > >>>> > > temporary image version tag. Then, in the
> > flink-statefun-playground,
> > > >>>> > change
> > > >>>> > > the image versions in the docker-compose files to use that
> > locally
> > > >>>> built
> > > >>>> > > image. IIRC, that's what we have been doing in the past.
> > > >>>> Admittedly, it's
> > > >>>> > > pretty manual - I don't think the CI manages this workflow.
> > > >>>> > >
> > > >>>> > > Thanks,
> > > >>>> > > Gordon
> > > >>>> > >
> > > >>>> > > On Mon, Aug 14, 2023 at 10:42 AM Galen Warren <
> > > >>>> galen@cvillewarrens.com>
> > > >>>> > > wrote:
> > > >>>> > >
> > > >>>> > > > I created a pull request for this: [FLINK-31619] Upgrade
> > Stateful
> > > >>>> > > > Functions to Flink 1.16.1 by galenwarren · Pull Request
> #331 ·
> > > >>>> > > > apache/flink-statefun (github.com)
> > > >>>> > > > <https://github.com/apache/flink-statefun/pull/331>.
> > > >>>> > > >
> > > >>>> > > > JIRA is here: [FLINK-31619] Upgrade Stateful Functions to
> > Flink
> > > >>>> 1.16.1
> > > >>>> > -
> > > >>>> > > > ASF JIRA (apache.org)
> > > >>>> > > > <
> https://issues.apache.org/jira/browse/FLINK-31619?filter=-1
> > >.
> > > >>>> > > >
> > > >>>> > > > Statefun references 1.16.2, despite the title -- that
> version
> > has
> > > >>>> come
> > > >>>> > > out
> > > >>>> > > > since the issue was created.
> > > >>>> > > >
> > > >>>> > > > I figured out how to run all the playground tests locally,
> > but it
> > > >>>> took
> > > >>>> > a
> > > >>>> > > > bit of reworking of the playground setup with respect to
> > Docker;
> > > >>>> > > > specifically, the Docker contexts used to build the example
> > > >>>> functions
> > > >>>> > > > needed to be broadened (i.e. moved up the tree) so that, if
> > > >>>> needed,
> > > >>>> > local
> > > >>>> > > > artifacts/code can be accessed from within the containers at
> > build
> > > >>>> > time.
> > > >>>> > > > Then I made the Docker compose.yml configurable through
> > > >>>> environment
> > > >>>> > > > variables to allow for them to run in either the original
> > manner
> > > >>>> --
> > > >>>> > i.e.
> > > >>>> > > > pulling artifacts from public repos -- or in a "local" mode,
> > where
> > > >>>> > > > artifacts are pulled from local builds.
> > > >>>> > > >
> > > >>>> > > > This process is a cleaner if the playground is a subfolder
> of
> > the
> > > >>>> > > > flink-statefun project rather than be its own repository
> > > >>>> > > > (flink-statefun-playground), because then all the relative
> > paths
> > > >>>> > between
> > > >>>> > > > the playground files and the build artifacts are fixed. So,
> > I'd
> > > >>>> like to
> > > >>>> > > > propose to move the playground files, modified as described
> > > >>>> above, to
> > > >>>> > > > flink-statefun/playground and retire
> > flink-statefun-playground. I
> > > >>>> can
> > > >>>> > > > submit separate PR s those changes if everyone is on board.
> > > >>>> > > >
> > > >>>> > > > Also, should I plan to do the same upgrade to handle Flink
> > > >>>> 1.17.x? It
> > > >>>> > > > should be easy to do, especially while the 1.16.x upgrade is
> > > >>>> fresh on
> > > >>>> > my
> > > >>>> > > > mind.
> > > >>>> > > >
> > > >>>> > > > Thanks.
> > > >>>> > > >
> > > >>>> > > >
> > > >>>> > > > On Fri, Aug 11, 2023 at 6:40 PM Galen Warren <
> > > >>>> galen@cvillewarrens.com>
> > > >>>> > > > wrote:
> > > >>>> > > >
> > > >>>> > > >> I'm done with the code to make Statefun compatible with
> Flink
> > > >>>> 1.16,
> > > >>>> > and
> > > >>>> > > >> all the tests (including e2e succeed). The required changes
> > were
> > > >>>> > pretty
> > > >>>> > > >> minimal.
> > > >>>> > > >>
> > > >>>> > > >> I'm running into a bit of a chicken/egg problem executing
> the
> > > >>>> tests in
> > > >>>> > > >> flink-statefun-playground
> > > >>>> > > >> <https://github.com/apache/flink-statefun-playground>,
> > though.
> > > >>>> That
> > > >>>> > > >> project seems to assume that all the various Statefun
> > artifacts
> > > >>>> are
> > > >>>> > > built
> > > >>>> > > >> and deployed to the various public repositories already.
> I've
> > > >>>> looked
> > > >>>> > > into
> > > >>>> > > >> trying to redirect references to local artifacts; however,
> > > >>>> that's also
> > > >>>> > > >> tricky since all the building occurs in Docker containers.
> > > >>>> > > >>
> > > >>>> > > >> Gordon, is there a trick to running the sample code in
> > > >>>> > > >> flink-statefun-playground against yet-unreleased code that
> > I'm
> > > >>>> > missing?
> > > >>>> > > >>
> > > >>>> > > >> Thanks.
> > > >>>> > > >>
> > > >>>> > > >> On Sat, Jun 24, 2023 at 12:40 PM Galen Warren <
> > > >>>> > galen@cvillewarrens.com>
> > > >>>> > > >> wrote:
> > > >>>> > > >>
> > > >>>> > > >>> Great -- thanks!
> > > >>>> > > >>>
> > > >>>> > > >>> I'm going to be out of town for about a week but I'll
> take a
> > > >>>> look at
> > > >>>> > > >>> this when I'm back.
> > > >>>> > > >>>
> > > >>>> > > >>> On Tue, Jun 20, 2023 at 8:46 AM Martijn Visser <
> > > >>>> mvisser@confluent.io
> > > >>>> > >
> > > >>>> > > >>> wrote:
> > > >>>> > > >>>
> > > >>>> > > >>>> Hi Galen,
> > > >>>> > > >>>>
> > > >>>> > > >>>> Yes, I'll be more than happy to help with Statefun
> > releases.
> > > >>>> > > >>>>
> > > >>>> > > >>>> Best regards,
> > > >>>> > > >>>>
> > > >>>> > > >>>> Martijn
> > > >>>> > > >>>>
> > > >>>> > > >>>> On Tue, Jun 20, 2023 at 2:21 PM Galen Warren <
> > > >>>> > galen@cvillewarrens.com
> > > >>>> > > >
> > > >>>> > > >>>> wrote:
> > > >>>> > > >>>>
> > > >>>> > > >>>>> Thanks.
> > > >>>> > > >>>>>
> > > >>>> > > >>>>> Martijn, to answer your question, I'd need to do a small
> > > >>>> amount of
> > > >>>> > > >>>>> work to get a PR ready, but not much. Happy to do it if
> > we're
> > > >>>> > > deciding to
> > > >>>> > > >>>>> restart Statefun releases -- are we?
> > > >>>> > > >>>>>
> > > >>>> > > >>>>> -- Galen
> > > >>>> > > >>>>>
> > > >>>> > > >>>>> On Sat, Jun 17, 2023 at 9:47 AM Tzu-Li (Gordon) Tai <
> > > >>>> > > >>>>> tzulitai@apache.org> wrote:
> > > >>>> > > >>>>>
> > > >>>> > > >>>>>> > Perhaps he could weigh in on whether the combination
> of
> > > >>>> > automated
> > > >>>> > > >>>>>> tests plus those smoke tests should be sufficient for
> > > >>>> testing with
> > > >>>> > > new
> > > >>>> > > >>>>>> Flink versions
> > > >>>> > > >>>>>>
> > > >>>> > > >>>>>> What we usually did at the bare minimum for new
> StateFun
> > > >>>> releases
> > > >>>> > > was
> > > >>>> > > >>>>>> the following:
> > > >>>> > > >>>>>>
> > > >>>> > > >>>>>>    1. Build tests (including the smoke tests in the e2e
> > > >>>> module,
> > > >>>> > > >>>>>>    which covers important tests like exactly-once
> > > >>>> verification)
> > > >>>> > > >>>>>>    2. Updating the flink-statefun-playground repo and
> > > >>>> manually
> > > >>>> > > >>>>>>    running all language examples there.
> > > >>>> > > >>>>>>
> > > >>>> > > >>>>>> If upgrading Flink versions was the only change in the
> > > >>>> release,
> > > >>>> > I'd
> > > >>>> > > >>>>>> probably say that this is sufficient.
> > > >>>> > > >>>>>>
> > > >>>> > > >>>>>> Best,
> > > >>>> > > >>>>>> Gordon
> > > >>>> > > >>>>>>
> > > >>>> > > >>>>>> On Thu, Jun 15, 2023 at 5:25 AM Martijn Visser <
> > > >>>> > > >>>>>> martijnvisser@apache.org> wrote:
> > > >>>> > > >>>>>>
> > > >>>> > > >>>>>>> Let me know if you have a PR for a Flink update :)
> > > >>>> > > >>>>>>>
> > > >>>> > > >>>>>>> On Thu, Jun 8, 2023 at 5:52 PM Galen Warren via user <
> > > >>>> > > >>>>>>> user@flink.apache.org> wrote:
> > > >>>> > > >>>>>>>
> > > >>>> > > >>>>>>>> Thanks Martijn.
> > > >>>> > > >>>>>>>>
> > > >>>> > > >>>>>>>> Personally, I'm already using a local fork of
> Statefun
> > > >>>> that is
> > > >>>> > > >>>>>>>> compatible with Flink 1.16.x, so I wouldn't have any
> > need
> > > >>>> for a
> > > >>>> > > released
> > > >>>> > > >>>>>>>> version compatible with 1.15.x. I'd be happy to do
> the
> > PRs
> > > >>>> to
> > > >>>> > > modify
> > > >>>> > > >>>>>>>> Statefun to work with new versions of Flink as they
> > come
> > > >>>> along.
> > > >>>> > > >>>>>>>>
> > > >>>> > > >>>>>>>> As for testing, Statefun does have unit tests and
> > Gordon
> > > >>>> also
> > > >>>> > sent
> > > >>>> > > >>>>>>>> me instructions a while back for how to do some
> > additional
> > > >>>> smoke
> > > >>>> > > tests
> > > >>>> > > >>>>>>>> which are pretty straightforward. Perhaps he could
> > weigh
> > > >>>> in on
> > > >>>> > > whether the
> > > >>>> > > >>>>>>>> combination of automated tests plus those smoke tests
> > > >>>> should be
> > > >>>> > > sufficient
> > > >>>> > > >>>>>>>> for testing with new Flink versions (I believe the
> > answer
> > > >>>> is
> > > >>>> > yes).
> > > >>>> > > >>>>>>>>
> > > >>>> > > >>>>>>>> -- Galen
> > > >>>> > > >>>>>>>>
> > > >>>> > > >>>>>>>>
> > > >>>> > > >>>>>>>>
> > > >>>> > > >>>>>>>> On Thu, Jun 8, 2023 at 8:01 AM Martijn Visser <
> > > >>>> > > >>>>>>>> martijnvisser@apache.org> wrote:
> > > >>>> > > >>>>>>>>
> > > >>>> > > >>>>>>>>> Hi all,
> > > >>>> > > >>>>>>>>>
> > > >>>> > > >>>>>>>>> Apologies for the late reply.
> > > >>>> > > >>>>>>>>>
> > > >>>> > > >>>>>>>>> I'm willing to help out with merging requests in
> > Statefun
> > > >>>> to
> > > >>>> > keep
> > > >>>> > > >>>>>>>>> them
> > > >>>> > > >>>>>>>>> compatible with new Flink releases and create new
> > > >>>> releases. I
> > > >>>> > do
> > > >>>> > > >>>>>>>>> think that
> > > >>>> > > >>>>>>>>> validation of the functionality of these releases
> > depends
> > > >>>> a lot
> > > >>>> > > on
> > > >>>> > > >>>>>>>>> those
> > > >>>> > > >>>>>>>>> who do these compatibility updates, with PMC members
> > > >>>> helping
> > > >>>> > out
> > > >>>> > > >>>>>>>>> with the
> > > >>>> > > >>>>>>>>> formal process.
> > > >>>> > > >>>>>>>>>
> > > >>>> > > >>>>>>>>> > Why can't the Apache Software Foundation allow
> > community
> > > >>>> > > members
> > > >>>> > > >>>>>>>>> to bring
> > > >>>> > > >>>>>>>>> it up to date?
> > > >>>> > > >>>>>>>>>
> > > >>>> > > >>>>>>>>> There's nothing preventing anyone from reviewing any
> > of
> > > >>>> the
> > > >>>> > > >>>>>>>>> current PRs or
> > > >>>> > > >>>>>>>>> opening new ones. However, none of them are approved
> > [1],
> > > >>>> so
> > > >>>> > > >>>>>>>>> there's also
> > > >>>> > > >>>>>>>>> nothing to merge.
> > > >>>> > > >>>>>>>>>
> > > >>>> > > >>>>>>>>> > I believe that there are people and companies on
> > this
> > > >>>> mailing
> > > >>>> > > >>>>>>>>> list
> > > >>>> > > >>>>>>>>> interested in supporting Apache Flink Stateful
> > Functions.
> > > >>>> > > >>>>>>>>>
> > > >>>> > > >>>>>>>>> If so, then now is the time to show.
> > > >>>> > > >>>>>>>>>
> > > >>>> > > >>>>>>>>> Would there be a preference to create a release with
> > > >>>> Galen's
> > > >>>> > > merged
> > > >>>> > > >>>>>>>>> compatibility update to Flink 1.15.2, or do we want
> to
> > > >>>> skip
> > > >>>> > that
> > > >>>> > > >>>>>>>>> and go
> > > >>>> > > >>>>>>>>> straight to a newer version?
> > > >>>> > > >>>>>>>>>
> > > >>>> > > >>>>>>>>> Best regards,
> > > >>>> > > >>>>>>>>>
> > > >>>> > > >>>>>>>>> Martijn
> > > >>>> > > >>>>>>>>>
> > > >>>> > > >>>>>>>>> [1]
> > > >>>> > > >>>>>>>>>
> > > >>>> > > >>>>>>>>>
> > > >>>> > >
> > > >>>> >
> > > >>>>
> >
> https://github.com/apache/flink-statefun/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved
> > > >>>> > > >>>>>>>>>
> > > >>>> > > >>>>>>>>> On Tue, Jun 6, 2023 at 3:55 PM Marco Villalobos <
> > > >>>> > > >>>>>>>>> mvillalobos@kineteque.com>
> > > >>>> > > >>>>>>>>> wrote:
> > > >>>> > > >>>>>>>>>
> > > >>>> > > >>>>>>>>> > Why can't the Apache Software Foundation allow
> > community
> > > >>>> > > members
> > > >>>> > > >>>>>>>>> to bring
> > > >>>> > > >>>>>>>>> > it up to date?
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > What's the process for that?
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > I believe that there are people and companies on
> > this
> > > >>>> mailing
> > > >>>> > > >>>>>>>>> list
> > > >>>> > > >>>>>>>>> > interested in supporting Apache Flink Stateful
> > > >>>> Functions.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > You already had two people on this thread express
> > > >>>> interest.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > At the very least, we could keep the library
> > versions
> > > >>>> up to
> > > >>>> > > date.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > There are only a small list of new features that
> > might
> > > >>>> be
> > > >>>> > > >>>>>>>>> worthwhile:
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > 1. event time processing
> > > >>>> > > >>>>>>>>> > 2. state rest api
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <
> > > >>>> > > chesnay@apache.org>
> > > >>>> > > >>>>>>>>> wrote:
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > If you were to fork it *and want to redistribute
> it*
> > > >>>> then the
> > > >>>> > > >>>>>>>>> short
> > > >>>> > > >>>>>>>>> > version is that
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> >    1. you have to adhere to the Apache licensing
> > > >>>> requirements
> > > >>>> > > >>>>>>>>> >    2. you have to make it clear that your fork
> does
> > not
> > > >>>> > belong
> > > >>>> > > >>>>>>>>> to the
> > > >>>> > > >>>>>>>>> >    Apache Flink project. (Trademarks and all that)
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > Neither should be significant hurdles (there
> should
> > > >>>> also be
> > > >>>> > > >>>>>>>>> plenty of
> > > >>>> > > >>>>>>>>> > online resources regarding 1), and if you do this
> > then
> > > >>>> you
> > > >>>> > can
> > > >>>> > > >>>>>>>>> freely share
> > > >>>> > > >>>>>>>>> > your fork with others.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > I've also pinged Martijn to take a look at this
> > thread.
> > > >>>> > > >>>>>>>>> > To my knowledge the project hasn't decided
> anything
> > yet.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > On 27/05/2023 04:05, Galen Warren wrote:
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > Ok, I get it. No interest.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > If this project is being abandoned, I guess I'll
> > work
> > > >>>> with my
> > > >>>> > > >>>>>>>>> own fork. Is
> > > >>>> > > >>>>>>>>> > there anything I should consider here? Can I share
> > it
> > > >>>> with
> > > >>>> > > other
> > > >>>> > > >>>>>>>>> people who
> > > >>>> > > >>>>>>>>> > use this project?
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > On Tue, May 16, 2023 at 10:50 AM Galen Warren <
> > > >>>> > > >>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
> > > >>>> > > >>>>>>>>> > wrote:
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > Hi Martijn, since you opened this discussion
> > thread, I'm
> > > >>>> > > curious
> > > >>>> > > >>>>>>>>> what your
> > > >>>> > > >>>>>>>>> > thoughts are in light of the responses? Thanks.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <
> > > >>>> > > >>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
> > > >>>> > > >>>>>>>>> > wrote:
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > I use Apache Flink for stream processing, and
> > StateFun
> > > >>>> as a
> > > >>>> > > >>>>>>>>> hand-off
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > point for the rest of the application.
> > > >>>> > > >>>>>>>>> > It serves well as a bridge between a Flink
> Streaming
> > > >>>> job and
> > > >>>> > > >>>>>>>>> > micro-services.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > This is essentially how I use it as well, and I
> > would
> > > >>>> also be
> > > >>>> > > >>>>>>>>> sad to see
> > > >>>> > > >>>>>>>>> > it sunsetted. It works well; I don't know that
> > there is
> > > >>>> a lot
> > > >>>> > > of
> > > >>>> > > >>>>>>>>> new
> > > >>>> > > >>>>>>>>> > development required, but if there are no new
> > Statefun
> > > >>>> > > releases,
> > > >>>> > > >>>>>>>>> then
> > > >>>> > > >>>>>>>>> > Statefun can only be used with older Flink
> versions.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos
> <
> > > >>>> > > >>>>>>>>> mvillalobos@kineteque.com> wrote:
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > I am currently using Stateful Functions in my
> > > >>>> application.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > I use Apache Flink for stream processing, and
> > StateFun
> > > >>>> as a
> > > >>>> > > >>>>>>>>> hand-off
> > > >>>> > > >>>>>>>>> > point for the rest of the application.
> > > >>>> > > >>>>>>>>> > It serves well as a bridge between a Flink
> Streaming
> > > >>>> job and
> > > >>>> > > >>>>>>>>> > micro-services.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > I would be disappointed if StateFun was sunsetted.
> > Its
> > > >>>> a
> > > >>>> > good
> > > >>>> > > >>>>>>>>> idea.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > If there is anything I can do to help, as a
> > contributor
> > > >>>> > > perhaps,
> > > >>>> > > >>>>>>>>> please
> > > >>>> > > >>>>>>>>> > let me know.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <
> > > >>>> > > >>>>>>>>> martijnvisser@apache.org> <martijnvisser@apache.org
> >
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > wrote:
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > Hi everyone,
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > I want to open a discussion on the status of the
> > > >>>> Statefun
> > > >>>> > > >>>>>>>>> Project [1]
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > in Apache Flink. As you might have noticed, there
> > > >>>> hasn't been
> > > >>>> > > >>>>>>>>> much
> > > >>>> > > >>>>>>>>> > development over the past months in the Statefun
> > > >>>> repository
> > > >>>> > > [2].
> > > >>>> > > >>>>>>>>> There is
> > > >>>> > > >>>>>>>>> > currently a lack of active contributors and
> > committers
> > > >>>> who
> > > >>>> > are
> > > >>>> > > >>>>>>>>> able to help
> > > >>>> > > >>>>>>>>> > with the maintenance of the project.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > In order to improve the situation, we need to
> solve
> > the
> > > >>>> lack
> > > >>>> > of
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > committers and the lack of contributors.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > On the lack of committers:
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > 1. Ideally, there are some of the current Flink
> > > >>>> committers
> > > >>>> > who
> > > >>>> > > >>>>>>>>> have
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > the bandwidth and can help with reviewing PRs and
> > > >>>> merging
> > > >>>> > them.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > 2. If that's not an option, it could be a
> > consideration
> > > >>>> that
> > > >>>> > > >>>>>>>>> current
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > committers only approve and review PRs, that are
> > > >>>> approved by
> > > >>>> > > >>>>>>>>> those who are
> > > >>>> > > >>>>>>>>> > willing to contribute to Statefun and if the CI
> > passes
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > On the lack of contributors:
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > 3. Next to having this discussion on the Dev and
> > User
> > > >>>> mailing
> > > >>>> > > >>>>>>>>> list, we
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > can also create a blog with a call for new
> > contributors
> > > >>>> on
> > > >>>> > the
> > > >>>> > > >>>>>>>>> Flink
> > > >>>> > > >>>>>>>>> > project website, send out some tweets on the
> Flink /
> > > >>>> Statefun
> > > >>>> > > >>>>>>>>> twitter
> > > >>>> > > >>>>>>>>> > accounts, post messages on Slack etc. In that
> > message,
> > > >>>> we
> > > >>>> > would
> > > >>>> > > >>>>>>>>> inform how
> > > >>>> > > >>>>>>>>> > those that are interested in contributing can
> start
> > and
> > > >>>> where
> > > >>>> > > >>>>>>>>> they could
> > > >>>> > > >>>>>>>>> > reach out for more information.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > There's also option 4. where a group of interested
> > > >>>> people
> > > >>>> > would
> > > >>>> > > >>>>>>>>> split
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > Statefun from the Flink project and make it a
> > separate
> > > >>>> top
> > > >>>> > > level
> > > >>>> > > >>>>>>>>> project
> > > >>>> > > >>>>>>>>> > under the Apache Flink umbrella (similar as
> > recently has
> > > >>>> > > >>>>>>>>> happened with
> > > >>>> > > >>>>>>>>> > Flink Table Store, which has become Apache
> Paimon).
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > If we see no improvements in the coming period, we
> > > >>>> should
> > > >>>> > > >>>>>>>>> consider
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > sunsetting Statefun and communicate that clearly
> to
> > the
> > > >>>> > users.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > I'm looking forward to your thoughts.
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > Best regards,
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > Martijn
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > [1]
> > > >>>> > > >>>>>>>>>
> > > >>>> https://nightlies.apache.org/flink/flink-statefun-docs-master/
> > > >>>> > <
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> >
> > > >>>> > https://nightlies.apache.org/flink/flink-statefun-docs-master/
> > > >>>> > > >
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > [2] https://github.com/apache/flink-statefun <
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> > https://github.com/apache/flink-statefun>
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>> >
> > > >>>> > > >>>>>>>>>
> > > >>>> > > >>>>>>>>
> > > >>>> > >
> > > >>>> >
> > > >>>>
> > > >>>
> >
>

Re: [DISCUSS] Status of Statefun Project

[Martijn Visser <ma...@apache.org>]

I forgot to include the link to this discussion thread for those that don't
have the history, that can be found at
https://lists.apache.org/thread/7cr2bgt91ppk6pz8o0nfbd10gs63nz6t

On Wed, Jan 28, 2026 at 12:28 PM Martijn Visser <ma...@apache.org>
wrote:

> Hi everyone,
>
> I'm bringing back this very old thread. It's been a while since we
> last discussed this, so I wanted to follow up on where things stand
> with Stateful Functions.
>
>  Since the 3.3.0 release in September 2023, there has been:
>  - Zero commits to the repository
>  - No new releases (now 4+ Flink versions behind: 1.16 vs 1.20/2.2)
>  - 22 open PRs unreviewed
>  - No sustained contributor activity
>
> I don't think that the current Flink committers and PMC members have
> the bandwidth or expertise in StateFun's codebase to maintain it.
> StateFun was always a specialized subproject maintained by a small
> group of dedicated contributors who have since moved on to other
> projects.
>
> Given this situation, I'd like to get the community's input:
>
> 1. Is anyone still actively using StateFun in production?
> 2. Should we consider sunsetting the project rather than leaving it in
> limbo?
>
> Thanks,
>
> Martijn
>
> Op wo 23 aug 2023 om 08:48 schreef Filip Karnicki <
> filip.karnicki@gmail.com>:
> >
> > Hi Gordon
> >
> > Any chance we could get this reviewed and released to the central repo?
> > We're currently forced to use a Flink version that has a nasty bug
> causing
> > an operational nightmare
> >
> > Many thanks
> > Fil
> >
> > On Sat, 19 Aug 2023 at 01:38, Galen Warren via user <
> user@flink.apache.org>
> > wrote:
> >
> > > Gotcha, makes sense as to the original division.
> > >
> > > >> Can this be solved by simply passing in the path to the artifacts
> > >
> > > This definitely works if we're going to be copying the artifacts on the
> > > host side -- into the build context -- and then from the context into
> the
> > > image. It only gets tricky to have a potentially varying path to the
> > > artifacts if we're trying to *directly *include the artifacts in the
> > > Docker context -- then we have a situation where the Docker context
> must
> > > contain both the artifacts and playground files, with (potentially)
> > > different root locations.
> > >
> > > Maybe the simplest thing to do here is just to leave the playground
> as-is
> > > and then copy the artifacts into the Docker context manually, prior to
> > > building the playground images. I'm fine with that. It will mean that
> each
> > > Statefun release will require two PRs and two sets of build/publish
> steps
> > > instead of one, but if everyone else is fine with that I am, too.
> Unless
> > > anyone objects, I'll go ahead and queue up a PR for the playground that
> > > makes these changes.
> > >
> > > Also, I should mention -- in case it's not clear -- that I have already
> > > built and run the playground examples with the code from the PR and
> > > everything worked. So that PR is ready to move forward with review,
> etc.,
> > > at this point.
> > >
> > > Thanks.
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Fri, Aug 18, 2023 at 4:16 PM Tzu-Li (Gordon) Tai <
> tzulitai@apache.org>
> > > wrote:
> > >
> > >> Hi Galen,
> > >>
> > >> The original intent of having a separate repo for the playground repo,
> > >> was that StateFun users can just go to that and start running simple
> > >> examples without any other distractions from the core code. I
> personally
> > >> don't have a strong preference here and can understand how it would
> make
> > >> the workflow more streamlined, but just FYI on the reasoning why are
> > >> separate in the first place.
> > >>
> > >> re: paths for locating StateFun artifacts.
> > >> Can this be solved by simply passing in the path to the artifacts? As
> > >> well as the image tag for the locally build base StateFun image. They
> could
> > >> probably be environment variables.
> > >>
> > >> Cheers,
> > >> Gordon
> > >>
> > >> On Fri, Aug 18, 2023 at 12:13 PM Galen Warren via user <
> > >> user@flink.apache.org> wrote:
> > >>
> > >>> Yes, exactly! And in addition to the base Statefun jars and the jar
> for
> > >>> the Java SDK, it does an equivalent copy/register operation for each
> of the
> > >>> other SDK libraries (Go, Python, Javascript) so that those libraries
> are
> > >>> also available when building the playground examples.
> > >>>
> > >>> One more question: In order to copy the various build artifacts into
> the
> > >>> Docker containers, those artifacts need to be part of the Docker
> context.
> > >>> With the playground being a separate project, that's slightly tricky
> to do,
> > >>> as there is no guarantee (other than convention) about the relative
> paths
> > >>> of *flink-statefun* and* flink-statefun-playground *in someone's
> local
> > >>> filesystem. The way I've set this up locally is to copy the
> playground into
> > >>> the* flink-statefun* project -- i.e. to *flink-statefun*/playground
> --
> > >>> such that I can set the Docker context to the root folder of
> > >>> *flink-statefun* and then have access to any local code and/or build
> > >>> artifacts.
> > >>>
> > >>> I'm wondering if there might be any appetite for making that move
> > >>> permanent, i.e. moving the playground to *flink-statefun*/playground
> > >>> and deprecating the standalone playground project. In addition to
> making
> > >>> the problem of building with unreleased artifacts a bit simpler to
> solve,
> > >>> it would also simplify the process of releasing a new Statefun
> version,
> > >>> since the entire process could be handled with a single PR and
> associated
> > >>> build/deploy tasks. In other words, a single PR could both update and
> > >>> deploy the Statefun package and the playground code and images.
> > >>>
> > >>> As it stands, at least two PRs would be required for each Statefun
> > >>> version update -- one for *flink-statefun* and one for
> > >>> *flink-statefun-playground*.
> > >>>
> > >>> Anyway, just an idea. Maybe there's an important reason for these
> > >>> projects to remain separate. If we do want to keep the playground
> project
> > >>> where it is, I could solve the copying problem by requiring the two
> > >>> projects to be siblings in the file system and by pre-copying the
> local
> > >>> build artifacts into a location accessible by the existing Docker
> contexts.
> > >>> This would still leave us with the need to have two PRs and releases
> > >>> instead of one, though.
> > >>>
> > >>> Thanks for your help!
> > >>>
> > >>>
> > >>> On Fri, Aug 18, 2023 at 2:45 PM Tzu-Li (Gordon) Tai <
> tzulitai@apache.org>
> > >>> wrote:
> > >>>
> > >>>> Hi Galen,
> > >>>>
> > >>>> > locally built code is copied into the build containers
> > >>>> so that it can be accessed during the build.
> > >>>>
> > >>>> That's exactly what we had been doing for release testing, yes.
> Sorry I
> > >>>> missed that detail in my previous response.
> > >>>>
> > >>>> And yes, that sounds like a reasonable approach. If I understand you
> > >>>> correctly, the workflow would become this:
> > >>>>
> > >>>>    1. Build the StateFun repo locally to install the snapshot
> artifact
> > >>>>    jars + have a local base StateFun image.
> > >>>>    2. Run the playground in "local" mode, so that it uses the local
> base
> > >>>>    StateFun image + builds the playground code using copied artifact
> > >>>> jars
> > >>>>    (instead of pulling from Maven).
> > >>>>
> > >>>> That looks good to me!
> > >>>>
> > >>>> Thanks,
> > >>>> Gordon
> > >>>>
> > >>>> On Fri, Aug 18, 2023 at 11:33 AM Galen Warren
> > >>>> <ga...@cvillewarrens.com.invalid> wrote:
> > >>>>
> > >>>> > Thanks.
> > >>>> >
> > >>>> > If you were to build a local image, as you suggest, how do you
> access
> > >>>> that
> > >>>> > image when building the playground images? All the compilation of
> > >>>> > playground code happens inside containers, so local images on the
> host
> > >>>> > aren't available in those containers. Unless I'm missing
> something?
> > >>>> >
> > >>>> > I've slightly reworked things such that the playground images can
> be
> > >>>> run in
> > >>>> > one of two modes -- the default mode, which works like before,
> and a
> > >>>> > "local" mode where locally built code is copied into the build
> > >>>> containers
> > >>>> > so that it can be accessed during the build. It works fine, you
> just
> > >>>> have
> > >>>> > to define a couple of environment variables when running
> > >>>> docker-compose to
> > >>>> > specify default vs. local mode and what versions of Flink and
> Statefun
> > >>>> > should be referenced, and then you can build a run the local
> examples
> > >>>> > without any additional steps. Does that sound like a reasonable
> > >>>> approach?
> > >>>> >
> > >>>> >
> > >>>> > On Fri, Aug 18, 2023 at 2:17 PM Tzu-Li (Gordon) Tai <
> > >>>> tzulitai@apache.org>
> > >>>> > wrote:
> > >>>> >
> > >>>> > > Hi Galen,
> > >>>> > >
> > >>>> > > > Gordon, is there a trick to running the sample code in
> > >>>> > > flink-statefun-playground against yet-unreleased code that I'm
> > >>>> missing?
> > >>>> > >
> > >>>> > > You'd have to locally build an image from the release branch,
> with a
> > >>>> > > temporary image version tag. Then, in the
> flink-statefun-playground,
> > >>>> > change
> > >>>> > > the image versions in the docker-compose files to use that
> locally
> > >>>> built
> > >>>> > > image. IIRC, that's what we have been doing in the past.
> > >>>> Admittedly, it's
> > >>>> > > pretty manual - I don't think the CI manages this workflow.
> > >>>> > >
> > >>>> > > Thanks,
> > >>>> > > Gordon
> > >>>> > >
> > >>>> > > On Mon, Aug 14, 2023 at 10:42 AM Galen Warren <
> > >>>> galen@cvillewarrens.com>
> > >>>> > > wrote:
> > >>>> > >
> > >>>> > > > I created a pull request for this: [FLINK-31619] Upgrade
> Stateful
> > >>>> > > > Functions to Flink 1.16.1 by galenwarren · Pull Request #331 ·
> > >>>> > > > apache/flink-statefun (github.com)
> > >>>> > > > <https://github.com/apache/flink-statefun/pull/331>.
> > >>>> > > >
> > >>>> > > > JIRA is here: [FLINK-31619] Upgrade Stateful Functions to
> Flink
> > >>>> 1.16.1
> > >>>> > -
> > >>>> > > > ASF JIRA (apache.org)
> > >>>> > > > <https://issues.apache.org/jira/browse/FLINK-31619?filter=-1
> >.
> > >>>> > > >
> > >>>> > > > Statefun references 1.16.2, despite the title -- that version
> has
> > >>>> come
> > >>>> > > out
> > >>>> > > > since the issue was created.
> > >>>> > > >
> > >>>> > > > I figured out how to run all the playground tests locally,
> but it
> > >>>> took
> > >>>> > a
> > >>>> > > > bit of reworking of the playground setup with respect to
> Docker;
> > >>>> > > > specifically, the Docker contexts used to build the example
> > >>>> functions
> > >>>> > > > needed to be broadened (i.e. moved up the tree) so that, if
> > >>>> needed,
> > >>>> > local
> > >>>> > > > artifacts/code can be accessed from within the containers at
> build
> > >>>> > time.
> > >>>> > > > Then I made the Docker compose.yml configurable through
> > >>>> environment
> > >>>> > > > variables to allow for them to run in either the original
> manner
> > >>>> --
> > >>>> > i.e.
> > >>>> > > > pulling artifacts from public repos -- or in a "local" mode,
> where
> > >>>> > > > artifacts are pulled from local builds.
> > >>>> > > >
> > >>>> > > > This process is a cleaner if the playground is a subfolder of
> the
> > >>>> > > > flink-statefun project rather than be its own repository
> > >>>> > > > (flink-statefun-playground), because then all the relative
> paths
> > >>>> > between
> > >>>> > > > the playground files and the build artifacts are fixed. So,
> I'd
> > >>>> like to
> > >>>> > > > propose to move the playground files, modified as described
> > >>>> above, to
> > >>>> > > > flink-statefun/playground and retire
> flink-statefun-playground. I
> > >>>> can
> > >>>> > > > submit separate PR s those changes if everyone is on board.
> > >>>> > > >
> > >>>> > > > Also, should I plan to do the same upgrade to handle Flink
> > >>>> 1.17.x? It
> > >>>> > > > should be easy to do, especially while the 1.16.x upgrade is
> > >>>> fresh on
> > >>>> > my
> > >>>> > > > mind.
> > >>>> > > >
> > >>>> > > > Thanks.
> > >>>> > > >
> > >>>> > > >
> > >>>> > > > On Fri, Aug 11, 2023 at 6:40 PM Galen Warren <
> > >>>> galen@cvillewarrens.com>
> > >>>> > > > wrote:
> > >>>> > > >
> > >>>> > > >> I'm done with the code to make Statefun compatible with Flink
> > >>>> 1.16,
> > >>>> > and
> > >>>> > > >> all the tests (including e2e succeed). The required changes
> were
> > >>>> > pretty
> > >>>> > > >> minimal.
> > >>>> > > >>
> > >>>> > > >> I'm running into a bit of a chicken/egg problem executing the
> > >>>> tests in
> > >>>> > > >> flink-statefun-playground
> > >>>> > > >> <https://github.com/apache/flink-statefun-playground>,
> though.
> > >>>> That
> > >>>> > > >> project seems to assume that all the various Statefun
> artifacts
> > >>>> are
> > >>>> > > built
> > >>>> > > >> and deployed to the various public repositories already. I've
> > >>>> looked
> > >>>> > > into
> > >>>> > > >> trying to redirect references to local artifacts; however,
> > >>>> that's also
> > >>>> > > >> tricky since all the building occurs in Docker containers.
> > >>>> > > >>
> > >>>> > > >> Gordon, is there a trick to running the sample code in
> > >>>> > > >> flink-statefun-playground against yet-unreleased code that
> I'm
> > >>>> > missing?
> > >>>> > > >>
> > >>>> > > >> Thanks.
> > >>>> > > >>
> > >>>> > > >> On Sat, Jun 24, 2023 at 12:40 PM Galen Warren <
> > >>>> > galen@cvillewarrens.com>
> > >>>> > > >> wrote:
> > >>>> > > >>
> > >>>> > > >>> Great -- thanks!
> > >>>> > > >>>
> > >>>> > > >>> I'm going to be out of town for about a week but I'll take a
> > >>>> look at
> > >>>> > > >>> this when I'm back.
> > >>>> > > >>>
> > >>>> > > >>> On Tue, Jun 20, 2023 at 8:46 AM Martijn Visser <
> > >>>> mvisser@confluent.io
> > >>>> > >
> > >>>> > > >>> wrote:
> > >>>> > > >>>
> > >>>> > > >>>> Hi Galen,
> > >>>> > > >>>>
> > >>>> > > >>>> Yes, I'll be more than happy to help with Statefun
> releases.
> > >>>> > > >>>>
> > >>>> > > >>>> Best regards,
> > >>>> > > >>>>
> > >>>> > > >>>> Martijn
> > >>>> > > >>>>
> > >>>> > > >>>> On Tue, Jun 20, 2023 at 2:21 PM Galen Warren <
> > >>>> > galen@cvillewarrens.com
> > >>>> > > >
> > >>>> > > >>>> wrote:
> > >>>> > > >>>>
> > >>>> > > >>>>> Thanks.
> > >>>> > > >>>>>
> > >>>> > > >>>>> Martijn, to answer your question, I'd need to do a small
> > >>>> amount of
> > >>>> > > >>>>> work to get a PR ready, but not much. Happy to do it if
> we're
> > >>>> > > deciding to
> > >>>> > > >>>>> restart Statefun releases -- are we?
> > >>>> > > >>>>>
> > >>>> > > >>>>> -- Galen
> > >>>> > > >>>>>
> > >>>> > > >>>>> On Sat, Jun 17, 2023 at 9:47 AM Tzu-Li (Gordon) Tai <
> > >>>> > > >>>>> tzulitai@apache.org> wrote:
> > >>>> > > >>>>>
> > >>>> > > >>>>>> > Perhaps he could weigh in on whether the combination of
> > >>>> > automated
> > >>>> > > >>>>>> tests plus those smoke tests should be sufficient for
> > >>>> testing with
> > >>>> > > new
> > >>>> > > >>>>>> Flink versions
> > >>>> > > >>>>>>
> > >>>> > > >>>>>> What we usually did at the bare minimum for new StateFun
> > >>>> releases
> > >>>> > > was
> > >>>> > > >>>>>> the following:
> > >>>> > > >>>>>>
> > >>>> > > >>>>>>    1. Build tests (including the smoke tests in the e2e
> > >>>> module,
> > >>>> > > >>>>>>    which covers important tests like exactly-once
> > >>>> verification)
> > >>>> > > >>>>>>    2. Updating the flink-statefun-playground repo and
> > >>>> manually
> > >>>> > > >>>>>>    running all language examples there.
> > >>>> > > >>>>>>
> > >>>> > > >>>>>> If upgrading Flink versions was the only change in the
> > >>>> release,
> > >>>> > I'd
> > >>>> > > >>>>>> probably say that this is sufficient.
> > >>>> > > >>>>>>
> > >>>> > > >>>>>> Best,
> > >>>> > > >>>>>> Gordon
> > >>>> > > >>>>>>
> > >>>> > > >>>>>> On Thu, Jun 15, 2023 at 5:25 AM Martijn Visser <
> > >>>> > > >>>>>> martijnvisser@apache.org> wrote:
> > >>>> > > >>>>>>
> > >>>> > > >>>>>>> Let me know if you have a PR for a Flink update :)
> > >>>> > > >>>>>>>
> > >>>> > > >>>>>>> On Thu, Jun 8, 2023 at 5:52 PM Galen Warren via user <
> > >>>> > > >>>>>>> user@flink.apache.org> wrote:
> > >>>> > > >>>>>>>
> > >>>> > > >>>>>>>> Thanks Martijn.
> > >>>> > > >>>>>>>>
> > >>>> > > >>>>>>>> Personally, I'm already using a local fork of Statefun
> > >>>> that is
> > >>>> > > >>>>>>>> compatible with Flink 1.16.x, so I wouldn't have any
> need
> > >>>> for a
> > >>>> > > released
> > >>>> > > >>>>>>>> version compatible with 1.15.x. I'd be happy to do the
> PRs
> > >>>> to
> > >>>> > > modify
> > >>>> > > >>>>>>>> Statefun to work with new versions of Flink as they
> come
> > >>>> along.
> > >>>> > > >>>>>>>>
> > >>>> > > >>>>>>>> As for testing, Statefun does have unit tests and
> Gordon
> > >>>> also
> > >>>> > sent
> > >>>> > > >>>>>>>> me instructions a while back for how to do some
> additional
> > >>>> smoke
> > >>>> > > tests
> > >>>> > > >>>>>>>> which are pretty straightforward. Perhaps he could
> weigh
> > >>>> in on
> > >>>> > > whether the
> > >>>> > > >>>>>>>> combination of automated tests plus those smoke tests
> > >>>> should be
> > >>>> > > sufficient
> > >>>> > > >>>>>>>> for testing with new Flink versions (I believe the
> answer
> > >>>> is
> > >>>> > yes).
> > >>>> > > >>>>>>>>
> > >>>> > > >>>>>>>> -- Galen
> > >>>> > > >>>>>>>>
> > >>>> > > >>>>>>>>
> > >>>> > > >>>>>>>>
> > >>>> > > >>>>>>>> On Thu, Jun 8, 2023 at 8:01 AM Martijn Visser <
> > >>>> > > >>>>>>>> martijnvisser@apache.org> wrote:
> > >>>> > > >>>>>>>>
> > >>>> > > >>>>>>>>> Hi all,
> > >>>> > > >>>>>>>>>
> > >>>> > > >>>>>>>>> Apologies for the late reply.
> > >>>> > > >>>>>>>>>
> > >>>> > > >>>>>>>>> I'm willing to help out with merging requests in
> Statefun
> > >>>> to
> > >>>> > keep
> > >>>> > > >>>>>>>>> them
> > >>>> > > >>>>>>>>> compatible with new Flink releases and create new
> > >>>> releases. I
> > >>>> > do
> > >>>> > > >>>>>>>>> think that
> > >>>> > > >>>>>>>>> validation of the functionality of these releases
> depends
> > >>>> a lot
> > >>>> > > on
> > >>>> > > >>>>>>>>> those
> > >>>> > > >>>>>>>>> who do these compatibility updates, with PMC members
> > >>>> helping
> > >>>> > out
> > >>>> > > >>>>>>>>> with the
> > >>>> > > >>>>>>>>> formal process.
> > >>>> > > >>>>>>>>>
> > >>>> > > >>>>>>>>> > Why can't the Apache Software Foundation allow
> community
> > >>>> > > members
> > >>>> > > >>>>>>>>> to bring
> > >>>> > > >>>>>>>>> it up to date?
> > >>>> > > >>>>>>>>>
> > >>>> > > >>>>>>>>> There's nothing preventing anyone from reviewing any
> of
> > >>>> the
> > >>>> > > >>>>>>>>> current PRs or
> > >>>> > > >>>>>>>>> opening new ones. However, none of them are approved
> [1],
> > >>>> so
> > >>>> > > >>>>>>>>> there's also
> > >>>> > > >>>>>>>>> nothing to merge.
> > >>>> > > >>>>>>>>>
> > >>>> > > >>>>>>>>> > I believe that there are people and companies on
> this
> > >>>> mailing
> > >>>> > > >>>>>>>>> list
> > >>>> > > >>>>>>>>> interested in supporting Apache Flink Stateful
> Functions.
> > >>>> > > >>>>>>>>>
> > >>>> > > >>>>>>>>> If so, then now is the time to show.
> > >>>> > > >>>>>>>>>
> > >>>> > > >>>>>>>>> Would there be a preference to create a release with
> > >>>> Galen's
> > >>>> > > merged
> > >>>> > > >>>>>>>>> compatibility update to Flink 1.15.2, or do we want to
> > >>>> skip
> > >>>> > that
> > >>>> > > >>>>>>>>> and go
> > >>>> > > >>>>>>>>> straight to a newer version?
> > >>>> > > >>>>>>>>>
> > >>>> > > >>>>>>>>> Best regards,
> > >>>> > > >>>>>>>>>
> > >>>> > > >>>>>>>>> Martijn
> > >>>> > > >>>>>>>>>
> > >>>> > > >>>>>>>>> [1]
> > >>>> > > >>>>>>>>>
> > >>>> > > >>>>>>>>>
> > >>>> > >
> > >>>> >
> > >>>>
> https://github.com/apache/flink-statefun/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved
> > >>>> > > >>>>>>>>>
> > >>>> > > >>>>>>>>> On Tue, Jun 6, 2023 at 3:55 PM Marco Villalobos <
> > >>>> > > >>>>>>>>> mvillalobos@kineteque.com>
> > >>>> > > >>>>>>>>> wrote:
> > >>>> > > >>>>>>>>>
> > >>>> > > >>>>>>>>> > Why can't the Apache Software Foundation allow
> community
> > >>>> > > members
> > >>>> > > >>>>>>>>> to bring
> > >>>> > > >>>>>>>>> > it up to date?
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > What's the process for that?
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > I believe that there are people and companies on
> this
> > >>>> mailing
> > >>>> > > >>>>>>>>> list
> > >>>> > > >>>>>>>>> > interested in supporting Apache Flink Stateful
> > >>>> Functions.
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > You already had two people on this thread express
> > >>>> interest.
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > At the very least, we could keep the library
> versions
> > >>>> up to
> > >>>> > > date.
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > There are only a small list of new features that
> might
> > >>>> be
> > >>>> > > >>>>>>>>> worthwhile:
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > 1. event time processing
> > >>>> > > >>>>>>>>> > 2. state rest api
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <
> > >>>> > > chesnay@apache.org>
> > >>>> > > >>>>>>>>> wrote:
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > If you were to fork it *and want to redistribute it*
> > >>>> then the
> > >>>> > > >>>>>>>>> short
> > >>>> > > >>>>>>>>> > version is that
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> >    1. you have to adhere to the Apache licensing
> > >>>> requirements
> > >>>> > > >>>>>>>>> >    2. you have to make it clear that your fork does
> not
> > >>>> > belong
> > >>>> > > >>>>>>>>> to the
> > >>>> > > >>>>>>>>> >    Apache Flink project. (Trademarks and all that)
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > Neither should be significant hurdles (there should
> > >>>> also be
> > >>>> > > >>>>>>>>> plenty of
> > >>>> > > >>>>>>>>> > online resources regarding 1), and if you do this
> then
> > >>>> you
> > >>>> > can
> > >>>> > > >>>>>>>>> freely share
> > >>>> > > >>>>>>>>> > your fork with others.
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > I've also pinged Martijn to take a look at this
> thread.
> > >>>> > > >>>>>>>>> > To my knowledge the project hasn't decided anything
> yet.
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > On 27/05/2023 04:05, Galen Warren wrote:
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > Ok, I get it. No interest.
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > If this project is being abandoned, I guess I'll
> work
> > >>>> with my
> > >>>> > > >>>>>>>>> own fork. Is
> > >>>> > > >>>>>>>>> > there anything I should consider here? Can I share
> it
> > >>>> with
> > >>>> > > other
> > >>>> > > >>>>>>>>> people who
> > >>>> > > >>>>>>>>> > use this project?
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > On Tue, May 16, 2023 at 10:50 AM Galen Warren <
> > >>>> > > >>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
> > >>>> > > >>>>>>>>> > wrote:
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > Hi Martijn, since you opened this discussion
> thread, I'm
> > >>>> > > curious
> > >>>> > > >>>>>>>>> what your
> > >>>> > > >>>>>>>>> > thoughts are in light of the responses? Thanks.
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <
> > >>>> > > >>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
> > >>>> > > >>>>>>>>> > wrote:
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > I use Apache Flink for stream processing, and
> StateFun
> > >>>> as a
> > >>>> > > >>>>>>>>> hand-off
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > point for the rest of the application.
> > >>>> > > >>>>>>>>> > It serves well as a bridge between a Flink Streaming
> > >>>> job and
> > >>>> > > >>>>>>>>> > micro-services.
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > This is essentially how I use it as well, and I
> would
> > >>>> also be
> > >>>> > > >>>>>>>>> sad to see
> > >>>> > > >>>>>>>>> > it sunsetted. It works well; I don't know that
> there is
> > >>>> a lot
> > >>>> > > of
> > >>>> > > >>>>>>>>> new
> > >>>> > > >>>>>>>>> > development required, but if there are no new
> Statefun
> > >>>> > > releases,
> > >>>> > > >>>>>>>>> then
> > >>>> > > >>>>>>>>> > Statefun can only be used with older Flink versions.
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
> > >>>> > > >>>>>>>>> mvillalobos@kineteque.com> wrote:
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > I am currently using Stateful Functions in my
> > >>>> application.
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > I use Apache Flink for stream processing, and
> StateFun
> > >>>> as a
> > >>>> > > >>>>>>>>> hand-off
> > >>>> > > >>>>>>>>> > point for the rest of the application.
> > >>>> > > >>>>>>>>> > It serves well as a bridge between a Flink Streaming
> > >>>> job and
> > >>>> > > >>>>>>>>> > micro-services.
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > I would be disappointed if StateFun was sunsetted.
> Its
> > >>>> a
> > >>>> > good
> > >>>> > > >>>>>>>>> idea.
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > If there is anything I can do to help, as a
> contributor
> > >>>> > > perhaps,
> > >>>> > > >>>>>>>>> please
> > >>>> > > >>>>>>>>> > let me know.
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <
> > >>>> > > >>>>>>>>> martijnvisser@apache.org> <ma...@apache.org>
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > wrote:
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > Hi everyone,
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > I want to open a discussion on the status of the
> > >>>> Statefun
> > >>>> > > >>>>>>>>> Project [1]
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > in Apache Flink. As you might have noticed, there
> > >>>> hasn't been
> > >>>> > > >>>>>>>>> much
> > >>>> > > >>>>>>>>> > development over the past months in the Statefun
> > >>>> repository
> > >>>> > > [2].
> > >>>> > > >>>>>>>>> There is
> > >>>> > > >>>>>>>>> > currently a lack of active contributors and
> committers
> > >>>> who
> > >>>> > are
> > >>>> > > >>>>>>>>> able to help
> > >>>> > > >>>>>>>>> > with the maintenance of the project.
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > In order to improve the situation, we need to solve
> the
> > >>>> lack
> > >>>> > of
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > committers and the lack of contributors.
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > On the lack of committers:
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > 1. Ideally, there are some of the current Flink
> > >>>> committers
> > >>>> > who
> > >>>> > > >>>>>>>>> have
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > the bandwidth and can help with reviewing PRs and
> > >>>> merging
> > >>>> > them.
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > 2. If that's not an option, it could be a
> consideration
> > >>>> that
> > >>>> > > >>>>>>>>> current
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > committers only approve and review PRs, that are
> > >>>> approved by
> > >>>> > > >>>>>>>>> those who are
> > >>>> > > >>>>>>>>> > willing to contribute to Statefun and if the CI
> passes
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > On the lack of contributors:
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > 3. Next to having this discussion on the Dev and
> User
> > >>>> mailing
> > >>>> > > >>>>>>>>> list, we
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > can also create a blog with a call for new
> contributors
> > >>>> on
> > >>>> > the
> > >>>> > > >>>>>>>>> Flink
> > >>>> > > >>>>>>>>> > project website, send out some tweets on the Flink /
> > >>>> Statefun
> > >>>> > > >>>>>>>>> twitter
> > >>>> > > >>>>>>>>> > accounts, post messages on Slack etc. In that
> message,
> > >>>> we
> > >>>> > would
> > >>>> > > >>>>>>>>> inform how
> > >>>> > > >>>>>>>>> > those that are interested in contributing can start
> and
> > >>>> where
> > >>>> > > >>>>>>>>> they could
> > >>>> > > >>>>>>>>> > reach out for more information.
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > There's also option 4. where a group of interested
> > >>>> people
> > >>>> > would
> > >>>> > > >>>>>>>>> split
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > Statefun from the Flink project and make it a
> separate
> > >>>> top
> > >>>> > > level
> > >>>> > > >>>>>>>>> project
> > >>>> > > >>>>>>>>> > under the Apache Flink umbrella (similar as
> recently has
> > >>>> > > >>>>>>>>> happened with
> > >>>> > > >>>>>>>>> > Flink Table Store, which has become Apache Paimon).
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > If we see no improvements in the coming period, we
> > >>>> should
> > >>>> > > >>>>>>>>> consider
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > sunsetting Statefun and communicate that clearly to
> the
> > >>>> > users.
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > I'm looking forward to your thoughts.
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > Best regards,
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > Martijn
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > [1]
> > >>>> > > >>>>>>>>>
> > >>>> https://nightlies.apache.org/flink/flink-statefun-docs-master/
> > >>>> > <
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> >
> > >>>> > https://nightlies.apache.org/flink/flink-statefun-docs-master/
> > >>>> > > >
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > [2] https://github.com/apache/flink-statefun <
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > https://github.com/apache/flink-statefun>
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>>
> > >>>> > > >>>>>>>>
> > >>>> > >
> > >>>> >
> > >>>>
> > >>>
>

Re: [DISCUSS] Status of Statefun Project

[Martijn Visser <ma...@apache.org>]

I forgot to include the link to this discussion thread for those that don't
have the history, that can be found at
https://lists.apache.org/thread/7cr2bgt91ppk6pz8o0nfbd10gs63nz6t

On Wed, Jan 28, 2026 at 12:28 PM Martijn Visser <ma...@apache.org>
wrote:

> Hi everyone,
>
> I'm bringing back this very old thread. It's been a while since we
> last discussed this, so I wanted to follow up on where things stand
> with Stateful Functions.
>
>  Since the 3.3.0 release in September 2023, there has been:
>  - Zero commits to the repository
>  - No new releases (now 4+ Flink versions behind: 1.16 vs 1.20/2.2)
>  - 22 open PRs unreviewed
>  - No sustained contributor activity
>
> I don't think that the current Flink committers and PMC members have
> the bandwidth or expertise in StateFun's codebase to maintain it.
> StateFun was always a specialized subproject maintained by a small
> group of dedicated contributors who have since moved on to other
> projects.
>
> Given this situation, I'd like to get the community's input:
>
> 1. Is anyone still actively using StateFun in production?
> 2. Should we consider sunsetting the project rather than leaving it in
> limbo?
>
> Thanks,
>
> Martijn
>
> Op wo 23 aug 2023 om 08:48 schreef Filip Karnicki <
> filip.karnicki@gmail.com>:
> >
> > Hi Gordon
> >
> > Any chance we could get this reviewed and released to the central repo?
> > We're currently forced to use a Flink version that has a nasty bug
> causing
> > an operational nightmare
> >
> > Many thanks
> > Fil
> >
> > On Sat, 19 Aug 2023 at 01:38, Galen Warren via user <
> user@flink.apache.org>
> > wrote:
> >
> > > Gotcha, makes sense as to the original division.
> > >
> > > >> Can this be solved by simply passing in the path to the artifacts
> > >
> > > This definitely works if we're going to be copying the artifacts on the
> > > host side -- into the build context -- and then from the context into
> the
> > > image. It only gets tricky to have a potentially varying path to the
> > > artifacts if we're trying to *directly *include the artifacts in the
> > > Docker context -- then we have a situation where the Docker context
> must
> > > contain both the artifacts and playground files, with (potentially)
> > > different root locations.
> > >
> > > Maybe the simplest thing to do here is just to leave the playground
> as-is
> > > and then copy the artifacts into the Docker context manually, prior to
> > > building the playground images. I'm fine with that. It will mean that
> each
> > > Statefun release will require two PRs and two sets of build/publish
> steps
> > > instead of one, but if everyone else is fine with that I am, too.
> Unless
> > > anyone objects, I'll go ahead and queue up a PR for the playground that
> > > makes these changes.
> > >
> > > Also, I should mention -- in case it's not clear -- that I have already
> > > built and run the playground examples with the code from the PR and
> > > everything worked. So that PR is ready to move forward with review,
> etc.,
> > > at this point.
> > >
> > > Thanks.
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Fri, Aug 18, 2023 at 4:16 PM Tzu-Li (Gordon) Tai <
> tzulitai@apache.org>
> > > wrote:
> > >
> > >> Hi Galen,
> > >>
> > >> The original intent of having a separate repo for the playground repo,
> > >> was that StateFun users can just go to that and start running simple
> > >> examples without any other distractions from the core code. I
> personally
> > >> don't have a strong preference here and can understand how it would
> make
> > >> the workflow more streamlined, but just FYI on the reasoning why are
> > >> separate in the first place.
> > >>
> > >> re: paths for locating StateFun artifacts.
> > >> Can this be solved by simply passing in the path to the artifacts? As
> > >> well as the image tag for the locally build base StateFun image. They
> could
> > >> probably be environment variables.
> > >>
> > >> Cheers,
> > >> Gordon
> > >>
> > >> On Fri, Aug 18, 2023 at 12:13 PM Galen Warren via user <
> > >> user@flink.apache.org> wrote:
> > >>
> > >>> Yes, exactly! And in addition to the base Statefun jars and the jar
> for
> > >>> the Java SDK, it does an equivalent copy/register operation for each
> of the
> > >>> other SDK libraries (Go, Python, Javascript) so that those libraries
> are
> > >>> also available when building the playground examples.
> > >>>
> > >>> One more question: In order to copy the various build artifacts into
> the
> > >>> Docker containers, those artifacts need to be part of the Docker
> context.
> > >>> With the playground being a separate project, that's slightly tricky
> to do,
> > >>> as there is no guarantee (other than convention) about the relative
> paths
> > >>> of *flink-statefun* and* flink-statefun-playground *in someone's
> local
> > >>> filesystem. The way I've set this up locally is to copy the
> playground into
> > >>> the* flink-statefun* project -- i.e. to *flink-statefun*/playground
> --
> > >>> such that I can set the Docker context to the root folder of
> > >>> *flink-statefun* and then have access to any local code and/or build
> > >>> artifacts.
> > >>>
> > >>> I'm wondering if there might be any appetite for making that move
> > >>> permanent, i.e. moving the playground to *flink-statefun*/playground
> > >>> and deprecating the standalone playground project. In addition to
> making
> > >>> the problem of building with unreleased artifacts a bit simpler to
> solve,
> > >>> it would also simplify the process of releasing a new Statefun
> version,
> > >>> since the entire process could be handled with a single PR and
> associated
> > >>> build/deploy tasks. In other words, a single PR could both update and
> > >>> deploy the Statefun package and the playground code and images.
> > >>>
> > >>> As it stands, at least two PRs would be required for each Statefun
> > >>> version update -- one for *flink-statefun* and one for
> > >>> *flink-statefun-playground*.
> > >>>
> > >>> Anyway, just an idea. Maybe there's an important reason for these
> > >>> projects to remain separate. If we do want to keep the playground
> project
> > >>> where it is, I could solve the copying problem by requiring the two
> > >>> projects to be siblings in the file system and by pre-copying the
> local
> > >>> build artifacts into a location accessible by the existing Docker
> contexts.
> > >>> This would still leave us with the need to have two PRs and releases
> > >>> instead of one, though.
> > >>>
> > >>> Thanks for your help!
> > >>>
> > >>>
> > >>> On Fri, Aug 18, 2023 at 2:45 PM Tzu-Li (Gordon) Tai <
> tzulitai@apache.org>
> > >>> wrote:
> > >>>
> > >>>> Hi Galen,
> > >>>>
> > >>>> > locally built code is copied into the build containers
> > >>>> so that it can be accessed during the build.
> > >>>>
> > >>>> That's exactly what we had been doing for release testing, yes.
> Sorry I
> > >>>> missed that detail in my previous response.
> > >>>>
> > >>>> And yes, that sounds like a reasonable approach. If I understand you
> > >>>> correctly, the workflow would become this:
> > >>>>
> > >>>>    1. Build the StateFun repo locally to install the snapshot
> artifact
> > >>>>    jars + have a local base StateFun image.
> > >>>>    2. Run the playground in "local" mode, so that it uses the local
> base
> > >>>>    StateFun image + builds the playground code using copied artifact
> > >>>> jars
> > >>>>    (instead of pulling from Maven).
> > >>>>
> > >>>> That looks good to me!
> > >>>>
> > >>>> Thanks,
> > >>>> Gordon
> > >>>>
> > >>>> On Fri, Aug 18, 2023 at 11:33 AM Galen Warren
> > >>>> <ga...@cvillewarrens.com.invalid> wrote:
> > >>>>
> > >>>> > Thanks.
> > >>>> >
> > >>>> > If you were to build a local image, as you suggest, how do you
> access
> > >>>> that
> > >>>> > image when building the playground images? All the compilation of
> > >>>> > playground code happens inside containers, so local images on the
> host
> > >>>> > aren't available in those containers. Unless I'm missing
> something?
> > >>>> >
> > >>>> > I've slightly reworked things such that the playground images can
> be
> > >>>> run in
> > >>>> > one of two modes -- the default mode, which works like before,
> and a
> > >>>> > "local" mode where locally built code is copied into the build
> > >>>> containers
> > >>>> > so that it can be accessed during the build. It works fine, you
> just
> > >>>> have
> > >>>> > to define a couple of environment variables when running
> > >>>> docker-compose to
> > >>>> > specify default vs. local mode and what versions of Flink and
> Statefun
> > >>>> > should be referenced, and then you can build a run the local
> examples
> > >>>> > without any additional steps. Does that sound like a reasonable
> > >>>> approach?
> > >>>> >
> > >>>> >
> > >>>> > On Fri, Aug 18, 2023 at 2:17 PM Tzu-Li (Gordon) Tai <
> > >>>> tzulitai@apache.org>
> > >>>> > wrote:
> > >>>> >
> > >>>> > > Hi Galen,
> > >>>> > >
> > >>>> > > > Gordon, is there a trick to running the sample code in
> > >>>> > > flink-statefun-playground against yet-unreleased code that I'm
> > >>>> missing?
> > >>>> > >
> > >>>> > > You'd have to locally build an image from the release branch,
> with a
> > >>>> > > temporary image version tag. Then, in the
> flink-statefun-playground,
> > >>>> > change
> > >>>> > > the image versions in the docker-compose files to use that
> locally
> > >>>> built
> > >>>> > > image. IIRC, that's what we have been doing in the past.
> > >>>> Admittedly, it's
> > >>>> > > pretty manual - I don't think the CI manages this workflow.
> > >>>> > >
> > >>>> > > Thanks,
> > >>>> > > Gordon
> > >>>> > >
> > >>>> > > On Mon, Aug 14, 2023 at 10:42 AM Galen Warren <
> > >>>> galen@cvillewarrens.com>
> > >>>> > > wrote:
> > >>>> > >
> > >>>> > > > I created a pull request for this: [FLINK-31619] Upgrade
> Stateful
> > >>>> > > > Functions to Flink 1.16.1 by galenwarren · Pull Request #331 ·
> > >>>> > > > apache/flink-statefun (github.com)
> > >>>> > > > <https://github.com/apache/flink-statefun/pull/331>.
> > >>>> > > >
> > >>>> > > > JIRA is here: [FLINK-31619] Upgrade Stateful Functions to
> Flink
> > >>>> 1.16.1
> > >>>> > -
> > >>>> > > > ASF JIRA (apache.org)
> > >>>> > > > <https://issues.apache.org/jira/browse/FLINK-31619?filter=-1
> >.
> > >>>> > > >
> > >>>> > > > Statefun references 1.16.2, despite the title -- that version
> has
> > >>>> come
> > >>>> > > out
> > >>>> > > > since the issue was created.
> > >>>> > > >
> > >>>> > > > I figured out how to run all the playground tests locally,
> but it
> > >>>> took
> > >>>> > a
> > >>>> > > > bit of reworking of the playground setup with respect to
> Docker;
> > >>>> > > > specifically, the Docker contexts used to build the example
> > >>>> functions
> > >>>> > > > needed to be broadened (i.e. moved up the tree) so that, if
> > >>>> needed,
> > >>>> > local
> > >>>> > > > artifacts/code can be accessed from within the containers at
> build
> > >>>> > time.
> > >>>> > > > Then I made the Docker compose.yml configurable through
> > >>>> environment
> > >>>> > > > variables to allow for them to run in either the original
> manner
> > >>>> --
> > >>>> > i.e.
> > >>>> > > > pulling artifacts from public repos -- or in a "local" mode,
> where
> > >>>> > > > artifacts are pulled from local builds.
> > >>>> > > >
> > >>>> > > > This process is a cleaner if the playground is a subfolder of
> the
> > >>>> > > > flink-statefun project rather than be its own repository
> > >>>> > > > (flink-statefun-playground), because then all the relative
> paths
> > >>>> > between
> > >>>> > > > the playground files and the build artifacts are fixed. So,
> I'd
> > >>>> like to
> > >>>> > > > propose to move the playground files, modified as described
> > >>>> above, to
> > >>>> > > > flink-statefun/playground and retire
> flink-statefun-playground. I
> > >>>> can
> > >>>> > > > submit separate PR s those changes if everyone is on board.
> > >>>> > > >
> > >>>> > > > Also, should I plan to do the same upgrade to handle Flink
> > >>>> 1.17.x? It
> > >>>> > > > should be easy to do, especially while the 1.16.x upgrade is
> > >>>> fresh on
> > >>>> > my
> > >>>> > > > mind.
> > >>>> > > >
> > >>>> > > > Thanks.
> > >>>> > > >
> > >>>> > > >
> > >>>> > > > On Fri, Aug 11, 2023 at 6:40 PM Galen Warren <
> > >>>> galen@cvillewarrens.com>
> > >>>> > > > wrote:
> > >>>> > > >
> > >>>> > > >> I'm done with the code to make Statefun compatible with Flink
> > >>>> 1.16,
> > >>>> > and
> > >>>> > > >> all the tests (including e2e succeed). The required changes
> were
> > >>>> > pretty
> > >>>> > > >> minimal.
> > >>>> > > >>
> > >>>> > > >> I'm running into a bit of a chicken/egg problem executing the
> > >>>> tests in
> > >>>> > > >> flink-statefun-playground
> > >>>> > > >> <https://github.com/apache/flink-statefun-playground>,
> though.
> > >>>> That
> > >>>> > > >> project seems to assume that all the various Statefun
> artifacts
> > >>>> are
> > >>>> > > built
> > >>>> > > >> and deployed to the various public repositories already. I've
> > >>>> looked
> > >>>> > > into
> > >>>> > > >> trying to redirect references to local artifacts; however,
> > >>>> that's also
> > >>>> > > >> tricky since all the building occurs in Docker containers.
> > >>>> > > >>
> > >>>> > > >> Gordon, is there a trick to running the sample code in
> > >>>> > > >> flink-statefun-playground against yet-unreleased code that
> I'm
> > >>>> > missing?
> > >>>> > > >>
> > >>>> > > >> Thanks.
> > >>>> > > >>
> > >>>> > > >> On Sat, Jun 24, 2023 at 12:40 PM Galen Warren <
> > >>>> > galen@cvillewarrens.com>
> > >>>> > > >> wrote:
> > >>>> > > >>
> > >>>> > > >>> Great -- thanks!
> > >>>> > > >>>
> > >>>> > > >>> I'm going to be out of town for about a week but I'll take a
> > >>>> look at
> > >>>> > > >>> this when I'm back.
> > >>>> > > >>>
> > >>>> > > >>> On Tue, Jun 20, 2023 at 8:46 AM Martijn Visser <
> > >>>> mvisser@confluent.io
> > >>>> > >
> > >>>> > > >>> wrote:
> > >>>> > > >>>
> > >>>> > > >>>> Hi Galen,
> > >>>> > > >>>>
> > >>>> > > >>>> Yes, I'll be more than happy to help with Statefun
> releases.
> > >>>> > > >>>>
> > >>>> > > >>>> Best regards,
> > >>>> > > >>>>
> > >>>> > > >>>> Martijn
> > >>>> > > >>>>
> > >>>> > > >>>> On Tue, Jun 20, 2023 at 2:21 PM Galen Warren <
> > >>>> > galen@cvillewarrens.com
> > >>>> > > >
> > >>>> > > >>>> wrote:
> > >>>> > > >>>>
> > >>>> > > >>>>> Thanks.
> > >>>> > > >>>>>
> > >>>> > > >>>>> Martijn, to answer your question, I'd need to do a small
> > >>>> amount of
> > >>>> > > >>>>> work to get a PR ready, but not much. Happy to do it if
> we're
> > >>>> > > deciding to
> > >>>> > > >>>>> restart Statefun releases -- are we?
> > >>>> > > >>>>>
> > >>>> > > >>>>> -- Galen
> > >>>> > > >>>>>
> > >>>> > > >>>>> On Sat, Jun 17, 2023 at 9:47 AM Tzu-Li (Gordon) Tai <
> > >>>> > > >>>>> tzulitai@apache.org> wrote:
> > >>>> > > >>>>>
> > >>>> > > >>>>>> > Perhaps he could weigh in on whether the combination of
> > >>>> > automated
> > >>>> > > >>>>>> tests plus those smoke tests should be sufficient for
> > >>>> testing with
> > >>>> > > new
> > >>>> > > >>>>>> Flink versions
> > >>>> > > >>>>>>
> > >>>> > > >>>>>> What we usually did at the bare minimum for new StateFun
> > >>>> releases
> > >>>> > > was
> > >>>> > > >>>>>> the following:
> > >>>> > > >>>>>>
> > >>>> > > >>>>>>    1. Build tests (including the smoke tests in the e2e
> > >>>> module,
> > >>>> > > >>>>>>    which covers important tests like exactly-once
> > >>>> verification)
> > >>>> > > >>>>>>    2. Updating the flink-statefun-playground repo and
> > >>>> manually
> > >>>> > > >>>>>>    running all language examples there.
> > >>>> > > >>>>>>
> > >>>> > > >>>>>> If upgrading Flink versions was the only change in the
> > >>>> release,
> > >>>> > I'd
> > >>>> > > >>>>>> probably say that this is sufficient.
> > >>>> > > >>>>>>
> > >>>> > > >>>>>> Best,
> > >>>> > > >>>>>> Gordon
> > >>>> > > >>>>>>
> > >>>> > > >>>>>> On Thu, Jun 15, 2023 at 5:25 AM Martijn Visser <
> > >>>> > > >>>>>> martijnvisser@apache.org> wrote:
> > >>>> > > >>>>>>
> > >>>> > > >>>>>>> Let me know if you have a PR for a Flink update :)
> > >>>> > > >>>>>>>
> > >>>> > > >>>>>>> On Thu, Jun 8, 2023 at 5:52 PM Galen Warren via user <
> > >>>> > > >>>>>>> user@flink.apache.org> wrote:
> > >>>> > > >>>>>>>
> > >>>> > > >>>>>>>> Thanks Martijn.
> > >>>> > > >>>>>>>>
> > >>>> > > >>>>>>>> Personally, I'm already using a local fork of Statefun
> > >>>> that is
> > >>>> > > >>>>>>>> compatible with Flink 1.16.x, so I wouldn't have any
> need
> > >>>> for a
> > >>>> > > released
> > >>>> > > >>>>>>>> version compatible with 1.15.x. I'd be happy to do the
> PRs
> > >>>> to
> > >>>> > > modify
> > >>>> > > >>>>>>>> Statefun to work with new versions of Flink as they
> come
> > >>>> along.
> > >>>> > > >>>>>>>>
> > >>>> > > >>>>>>>> As for testing, Statefun does have unit tests and
> Gordon
> > >>>> also
> > >>>> > sent
> > >>>> > > >>>>>>>> me instructions a while back for how to do some
> additional
> > >>>> smoke
> > >>>> > > tests
> > >>>> > > >>>>>>>> which are pretty straightforward. Perhaps he could
> weigh
> > >>>> in on
> > >>>> > > whether the
> > >>>> > > >>>>>>>> combination of automated tests plus those smoke tests
> > >>>> should be
> > >>>> > > sufficient
> > >>>> > > >>>>>>>> for testing with new Flink versions (I believe the
> answer
> > >>>> is
> > >>>> > yes).
> > >>>> > > >>>>>>>>
> > >>>> > > >>>>>>>> -- Galen
> > >>>> > > >>>>>>>>
> > >>>> > > >>>>>>>>
> > >>>> > > >>>>>>>>
> > >>>> > > >>>>>>>> On Thu, Jun 8, 2023 at 8:01 AM Martijn Visser <
> > >>>> > > >>>>>>>> martijnvisser@apache.org> wrote:
> > >>>> > > >>>>>>>>
> > >>>> > > >>>>>>>>> Hi all,
> > >>>> > > >>>>>>>>>
> > >>>> > > >>>>>>>>> Apologies for the late reply.
> > >>>> > > >>>>>>>>>
> > >>>> > > >>>>>>>>> I'm willing to help out with merging requests in
> Statefun
> > >>>> to
> > >>>> > keep
> > >>>> > > >>>>>>>>> them
> > >>>> > > >>>>>>>>> compatible with new Flink releases and create new
> > >>>> releases. I
> > >>>> > do
> > >>>> > > >>>>>>>>> think that
> > >>>> > > >>>>>>>>> validation of the functionality of these releases
> depends
> > >>>> a lot
> > >>>> > > on
> > >>>> > > >>>>>>>>> those
> > >>>> > > >>>>>>>>> who do these compatibility updates, with PMC members
> > >>>> helping
> > >>>> > out
> > >>>> > > >>>>>>>>> with the
> > >>>> > > >>>>>>>>> formal process.
> > >>>> > > >>>>>>>>>
> > >>>> > > >>>>>>>>> > Why can't the Apache Software Foundation allow
> community
> > >>>> > > members
> > >>>> > > >>>>>>>>> to bring
> > >>>> > > >>>>>>>>> it up to date?
> > >>>> > > >>>>>>>>>
> > >>>> > > >>>>>>>>> There's nothing preventing anyone from reviewing any
> of
> > >>>> the
> > >>>> > > >>>>>>>>> current PRs or
> > >>>> > > >>>>>>>>> opening new ones. However, none of them are approved
> [1],
> > >>>> so
> > >>>> > > >>>>>>>>> there's also
> > >>>> > > >>>>>>>>> nothing to merge.
> > >>>> > > >>>>>>>>>
> > >>>> > > >>>>>>>>> > I believe that there are people and companies on
> this
> > >>>> mailing
> > >>>> > > >>>>>>>>> list
> > >>>> > > >>>>>>>>> interested in supporting Apache Flink Stateful
> Functions.
> > >>>> > > >>>>>>>>>
> > >>>> > > >>>>>>>>> If so, then now is the time to show.
> > >>>> > > >>>>>>>>>
> > >>>> > > >>>>>>>>> Would there be a preference to create a release with
> > >>>> Galen's
> > >>>> > > merged
> > >>>> > > >>>>>>>>> compatibility update to Flink 1.15.2, or do we want to
> > >>>> skip
> > >>>> > that
> > >>>> > > >>>>>>>>> and go
> > >>>> > > >>>>>>>>> straight to a newer version?
> > >>>> > > >>>>>>>>>
> > >>>> > > >>>>>>>>> Best regards,
> > >>>> > > >>>>>>>>>
> > >>>> > > >>>>>>>>> Martijn
> > >>>> > > >>>>>>>>>
> > >>>> > > >>>>>>>>> [1]
> > >>>> > > >>>>>>>>>
> > >>>> > > >>>>>>>>>
> > >>>> > >
> > >>>> >
> > >>>>
> https://github.com/apache/flink-statefun/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved
> > >>>> > > >>>>>>>>>
> > >>>> > > >>>>>>>>> On Tue, Jun 6, 2023 at 3:55 PM Marco Villalobos <
> > >>>> > > >>>>>>>>> mvillalobos@kineteque.com>
> > >>>> > > >>>>>>>>> wrote:
> > >>>> > > >>>>>>>>>
> > >>>> > > >>>>>>>>> > Why can't the Apache Software Foundation allow
> community
> > >>>> > > members
> > >>>> > > >>>>>>>>> to bring
> > >>>> > > >>>>>>>>> > it up to date?
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > What's the process for that?
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > I believe that there are people and companies on
> this
> > >>>> mailing
> > >>>> > > >>>>>>>>> list
> > >>>> > > >>>>>>>>> > interested in supporting Apache Flink Stateful
> > >>>> Functions.
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > You already had two people on this thread express
> > >>>> interest.
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > At the very least, we could keep the library
> versions
> > >>>> up to
> > >>>> > > date.
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > There are only a small list of new features that
> might
> > >>>> be
> > >>>> > > >>>>>>>>> worthwhile:
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > 1. event time processing
> > >>>> > > >>>>>>>>> > 2. state rest api
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <
> > >>>> > > chesnay@apache.org>
> > >>>> > > >>>>>>>>> wrote:
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > If you were to fork it *and want to redistribute it*
> > >>>> then the
> > >>>> > > >>>>>>>>> short
> > >>>> > > >>>>>>>>> > version is that
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> >    1. you have to adhere to the Apache licensing
> > >>>> requirements
> > >>>> > > >>>>>>>>> >    2. you have to make it clear that your fork does
> not
> > >>>> > belong
> > >>>> > > >>>>>>>>> to the
> > >>>> > > >>>>>>>>> >    Apache Flink project. (Trademarks and all that)
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > Neither should be significant hurdles (there should
> > >>>> also be
> > >>>> > > >>>>>>>>> plenty of
> > >>>> > > >>>>>>>>> > online resources regarding 1), and if you do this
> then
> > >>>> you
> > >>>> > can
> > >>>> > > >>>>>>>>> freely share
> > >>>> > > >>>>>>>>> > your fork with others.
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > I've also pinged Martijn to take a look at this
> thread.
> > >>>> > > >>>>>>>>> > To my knowledge the project hasn't decided anything
> yet.
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > On 27/05/2023 04:05, Galen Warren wrote:
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > Ok, I get it. No interest.
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > If this project is being abandoned, I guess I'll
> work
> > >>>> with my
> > >>>> > > >>>>>>>>> own fork. Is
> > >>>> > > >>>>>>>>> > there anything I should consider here? Can I share
> it
> > >>>> with
> > >>>> > > other
> > >>>> > > >>>>>>>>> people who
> > >>>> > > >>>>>>>>> > use this project?
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > On Tue, May 16, 2023 at 10:50 AM Galen Warren <
> > >>>> > > >>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
> > >>>> > > >>>>>>>>> > wrote:
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > Hi Martijn, since you opened this discussion
> thread, I'm
> > >>>> > > curious
> > >>>> > > >>>>>>>>> what your
> > >>>> > > >>>>>>>>> > thoughts are in light of the responses? Thanks.
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <
> > >>>> > > >>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
> > >>>> > > >>>>>>>>> > wrote:
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > I use Apache Flink for stream processing, and
> StateFun
> > >>>> as a
> > >>>> > > >>>>>>>>> hand-off
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > point for the rest of the application.
> > >>>> > > >>>>>>>>> > It serves well as a bridge between a Flink Streaming
> > >>>> job and
> > >>>> > > >>>>>>>>> > micro-services.
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > This is essentially how I use it as well, and I
> would
> > >>>> also be
> > >>>> > > >>>>>>>>> sad to see
> > >>>> > > >>>>>>>>> > it sunsetted. It works well; I don't know that
> there is
> > >>>> a lot
> > >>>> > > of
> > >>>> > > >>>>>>>>> new
> > >>>> > > >>>>>>>>> > development required, but if there are no new
> Statefun
> > >>>> > > releases,
> > >>>> > > >>>>>>>>> then
> > >>>> > > >>>>>>>>> > Statefun can only be used with older Flink versions.
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
> > >>>> > > >>>>>>>>> mvillalobos@kineteque.com> wrote:
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > I am currently using Stateful Functions in my
> > >>>> application.
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > I use Apache Flink for stream processing, and
> StateFun
> > >>>> as a
> > >>>> > > >>>>>>>>> hand-off
> > >>>> > > >>>>>>>>> > point for the rest of the application.
> > >>>> > > >>>>>>>>> > It serves well as a bridge between a Flink Streaming
> > >>>> job and
> > >>>> > > >>>>>>>>> > micro-services.
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > I would be disappointed if StateFun was sunsetted.
> Its
> > >>>> a
> > >>>> > good
> > >>>> > > >>>>>>>>> idea.
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > If there is anything I can do to help, as a
> contributor
> > >>>> > > perhaps,
> > >>>> > > >>>>>>>>> please
> > >>>> > > >>>>>>>>> > let me know.
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <
> > >>>> > > >>>>>>>>> martijnvisser@apache.org> <ma...@apache.org>
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > wrote:
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > Hi everyone,
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > I want to open a discussion on the status of the
> > >>>> Statefun
> > >>>> > > >>>>>>>>> Project [1]
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > in Apache Flink. As you might have noticed, there
> > >>>> hasn't been
> > >>>> > > >>>>>>>>> much
> > >>>> > > >>>>>>>>> > development over the past months in the Statefun
> > >>>> repository
> > >>>> > > [2].
> > >>>> > > >>>>>>>>> There is
> > >>>> > > >>>>>>>>> > currently a lack of active contributors and
> committers
> > >>>> who
> > >>>> > are
> > >>>> > > >>>>>>>>> able to help
> > >>>> > > >>>>>>>>> > with the maintenance of the project.
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > In order to improve the situation, we need to solve
> the
> > >>>> lack
> > >>>> > of
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > committers and the lack of contributors.
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > On the lack of committers:
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > 1. Ideally, there are some of the current Flink
> > >>>> committers
> > >>>> > who
> > >>>> > > >>>>>>>>> have
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > the bandwidth and can help with reviewing PRs and
> > >>>> merging
> > >>>> > them.
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > 2. If that's not an option, it could be a
> consideration
> > >>>> that
> > >>>> > > >>>>>>>>> current
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > committers only approve and review PRs, that are
> > >>>> approved by
> > >>>> > > >>>>>>>>> those who are
> > >>>> > > >>>>>>>>> > willing to contribute to Statefun and if the CI
> passes
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > On the lack of contributors:
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > 3. Next to having this discussion on the Dev and
> User
> > >>>> mailing
> > >>>> > > >>>>>>>>> list, we
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > can also create a blog with a call for new
> contributors
> > >>>> on
> > >>>> > the
> > >>>> > > >>>>>>>>> Flink
> > >>>> > > >>>>>>>>> > project website, send out some tweets on the Flink /
> > >>>> Statefun
> > >>>> > > >>>>>>>>> twitter
> > >>>> > > >>>>>>>>> > accounts, post messages on Slack etc. In that
> message,
> > >>>> we
> > >>>> > would
> > >>>> > > >>>>>>>>> inform how
> > >>>> > > >>>>>>>>> > those that are interested in contributing can start
> and
> > >>>> where
> > >>>> > > >>>>>>>>> they could
> > >>>> > > >>>>>>>>> > reach out for more information.
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > There's also option 4. where a group of interested
> > >>>> people
> > >>>> > would
> > >>>> > > >>>>>>>>> split
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > Statefun from the Flink project and make it a
> separate
> > >>>> top
> > >>>> > > level
> > >>>> > > >>>>>>>>> project
> > >>>> > > >>>>>>>>> > under the Apache Flink umbrella (similar as
> recently has
> > >>>> > > >>>>>>>>> happened with
> > >>>> > > >>>>>>>>> > Flink Table Store, which has become Apache Paimon).
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > If we see no improvements in the coming period, we
> > >>>> should
> > >>>> > > >>>>>>>>> consider
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > sunsetting Statefun and communicate that clearly to
> the
> > >>>> > users.
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > I'm looking forward to your thoughts.
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > Best regards,
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > Martijn
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > [1]
> > >>>> > > >>>>>>>>>
> > >>>> https://nightlies.apache.org/flink/flink-statefun-docs-master/
> > >>>> > <
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> >
> > >>>> > https://nightlies.apache.org/flink/flink-statefun-docs-master/
> > >>>> > > >
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > [2] https://github.com/apache/flink-statefun <
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> > https://github.com/apache/flink-statefun>
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>> >
> > >>>> > > >>>>>>>>>
> > >>>> > > >>>>>>>>
> > >>>> > >
> > >>>> >
> > >>>>
> > >>>
>

Re: [DISCUSS] Status of Statefun Project

[Martijn Visser <ma...@apache.org>]

Hi everyone,

I'm bringing back this very old thread. It's been a while since we
last discussed this, so I wanted to follow up on where things stand
with Stateful Functions.

 Since the 3.3.0 release in September 2023, there has been:
 - Zero commits to the repository
 - No new releases (now 4+ Flink versions behind: 1.16 vs 1.20/2.2)
 - 22 open PRs unreviewed
 - No sustained contributor activity

I don't think that the current Flink committers and PMC members have
the bandwidth or expertise in StateFun's codebase to maintain it.
StateFun was always a specialized subproject maintained by a small
group of dedicated contributors who have since moved on to other
projects.

Given this situation, I'd like to get the community's input:

1. Is anyone still actively using StateFun in production?
2. Should we consider sunsetting the project rather than leaving it in limbo?

Thanks,

Martijn

Op wo 23 aug 2023 om 08:48 schreef Filip Karnicki <fi...@gmail.com>:
>
> Hi Gordon
>
> Any chance we could get this reviewed and released to the central repo?
> We're currently forced to use a Flink version that has a nasty bug causing
> an operational nightmare
>
> Many thanks
> Fil
>
> On Sat, 19 Aug 2023 at 01:38, Galen Warren via user <us...@flink.apache.org>
> wrote:
>
> > Gotcha, makes sense as to the original division.
> >
> > >> Can this be solved by simply passing in the path to the artifacts
> >
> > This definitely works if we're going to be copying the artifacts on the
> > host side -- into the build context -- and then from the context into the
> > image. It only gets tricky to have a potentially varying path to the
> > artifacts if we're trying to *directly *include the artifacts in the
> > Docker context -- then we have a situation where the Docker context must
> > contain both the artifacts and playground files, with (potentially)
> > different root locations.
> >
> > Maybe the simplest thing to do here is just to leave the playground as-is
> > and then copy the artifacts into the Docker context manually, prior to
> > building the playground images. I'm fine with that. It will mean that each
> > Statefun release will require two PRs and two sets of build/publish steps
> > instead of one, but if everyone else is fine with that I am, too. Unless
> > anyone objects, I'll go ahead and queue up a PR for the playground that
> > makes these changes.
> >
> > Also, I should mention -- in case it's not clear -- that I have already
> > built and run the playground examples with the code from the PR and
> > everything worked. So that PR is ready to move forward with review, etc.,
> > at this point.
> >
> > Thanks.
> >
> >
> >
> >
> >
> >
> >
> > On Fri, Aug 18, 2023 at 4:16 PM Tzu-Li (Gordon) Tai <tz...@apache.org>
> > wrote:
> >
> >> Hi Galen,
> >>
> >> The original intent of having a separate repo for the playground repo,
> >> was that StateFun users can just go to that and start running simple
> >> examples without any other distractions from the core code. I personally
> >> don't have a strong preference here and can understand how it would make
> >> the workflow more streamlined, but just FYI on the reasoning why are
> >> separate in the first place.
> >>
> >> re: paths for locating StateFun artifacts.
> >> Can this be solved by simply passing in the path to the artifacts? As
> >> well as the image tag for the locally build base StateFun image. They could
> >> probably be environment variables.
> >>
> >> Cheers,
> >> Gordon
> >>
> >> On Fri, Aug 18, 2023 at 12:13 PM Galen Warren via user <
> >> user@flink.apache.org> wrote:
> >>
> >>> Yes, exactly! And in addition to the base Statefun jars and the jar for
> >>> the Java SDK, it does an equivalent copy/register operation for each of the
> >>> other SDK libraries (Go, Python, Javascript) so that those libraries are
> >>> also available when building the playground examples.
> >>>
> >>> One more question: In order to copy the various build artifacts into the
> >>> Docker containers, those artifacts need to be part of the Docker context.
> >>> With the playground being a separate project, that's slightly tricky to do,
> >>> as there is no guarantee (other than convention) about the relative paths
> >>> of *flink-statefun* and* flink-statefun-playground *in someone's local
> >>> filesystem. The way I've set this up locally is to copy the playground into
> >>> the* flink-statefun* project -- i.e. to *flink-statefun*/playground --
> >>> such that I can set the Docker context to the root folder of
> >>> *flink-statefun* and then have access to any local code and/or build
> >>> artifacts.
> >>>
> >>> I'm wondering if there might be any appetite for making that move
> >>> permanent, i.e. moving the playground to *flink-statefun*/playground
> >>> and deprecating the standalone playground project. In addition to making
> >>> the problem of building with unreleased artifacts a bit simpler to solve,
> >>> it would also simplify the process of releasing a new Statefun version,
> >>> since the entire process could be handled with a single PR and associated
> >>> build/deploy tasks. In other words, a single PR could both update and
> >>> deploy the Statefun package and the playground code and images.
> >>>
> >>> As it stands, at least two PRs would be required for each Statefun
> >>> version update -- one for *flink-statefun* and one for
> >>> *flink-statefun-playground*.
> >>>
> >>> Anyway, just an idea. Maybe there's an important reason for these
> >>> projects to remain separate. If we do want to keep the playground project
> >>> where it is, I could solve the copying problem by requiring the two
> >>> projects to be siblings in the file system and by pre-copying the local
> >>> build artifacts into a location accessible by the existing Docker contexts.
> >>> This would still leave us with the need to have two PRs and releases
> >>> instead of one, though.
> >>>
> >>> Thanks for your help!
> >>>
> >>>
> >>> On Fri, Aug 18, 2023 at 2:45 PM Tzu-Li (Gordon) Tai <tz...@apache.org>
> >>> wrote:
> >>>
> >>>> Hi Galen,
> >>>>
> >>>> > locally built code is copied into the build containers
> >>>> so that it can be accessed during the build.
> >>>>
> >>>> That's exactly what we had been doing for release testing, yes. Sorry I
> >>>> missed that detail in my previous response.
> >>>>
> >>>> And yes, that sounds like a reasonable approach. If I understand you
> >>>> correctly, the workflow would become this:
> >>>>
> >>>>    1. Build the StateFun repo locally to install the snapshot artifact
> >>>>    jars + have a local base StateFun image.
> >>>>    2. Run the playground in "local" mode, so that it uses the local base
> >>>>    StateFun image + builds the playground code using copied artifact
> >>>> jars
> >>>>    (instead of pulling from Maven).
> >>>>
> >>>> That looks good to me!
> >>>>
> >>>> Thanks,
> >>>> Gordon
> >>>>
> >>>> On Fri, Aug 18, 2023 at 11:33 AM Galen Warren
> >>>> <ga...@cvillewarrens.com.invalid> wrote:
> >>>>
> >>>> > Thanks.
> >>>> >
> >>>> > If you were to build a local image, as you suggest, how do you access
> >>>> that
> >>>> > image when building the playground images? All the compilation of
> >>>> > playground code happens inside containers, so local images on the host
> >>>> > aren't available in those containers. Unless I'm missing something?
> >>>> >
> >>>> > I've slightly reworked things such that the playground images can be
> >>>> run in
> >>>> > one of two modes -- the default mode, which works like before, and a
> >>>> > "local" mode where locally built code is copied into the build
> >>>> containers
> >>>> > so that it can be accessed during the build. It works fine, you just
> >>>> have
> >>>> > to define a couple of environment variables when running
> >>>> docker-compose to
> >>>> > specify default vs. local mode and what versions of Flink and Statefun
> >>>> > should be referenced, and then you can build a run the local examples
> >>>> > without any additional steps. Does that sound like a reasonable
> >>>> approach?
> >>>> >
> >>>> >
> >>>> > On Fri, Aug 18, 2023 at 2:17 PM Tzu-Li (Gordon) Tai <
> >>>> tzulitai@apache.org>
> >>>> > wrote:
> >>>> >
> >>>> > > Hi Galen,
> >>>> > >
> >>>> > > > Gordon, is there a trick to running the sample code in
> >>>> > > flink-statefun-playground against yet-unreleased code that I'm
> >>>> missing?
> >>>> > >
> >>>> > > You'd have to locally build an image from the release branch, with a
> >>>> > > temporary image version tag. Then, in the flink-statefun-playground,
> >>>> > change
> >>>> > > the image versions in the docker-compose files to use that locally
> >>>> built
> >>>> > > image. IIRC, that's what we have been doing in the past.
> >>>> Admittedly, it's
> >>>> > > pretty manual - I don't think the CI manages this workflow.
> >>>> > >
> >>>> > > Thanks,
> >>>> > > Gordon
> >>>> > >
> >>>> > > On Mon, Aug 14, 2023 at 10:42 AM Galen Warren <
> >>>> galen@cvillewarrens.com>
> >>>> > > wrote:
> >>>> > >
> >>>> > > > I created a pull request for this: [FLINK-31619] Upgrade Stateful
> >>>> > > > Functions to Flink 1.16.1 by galenwarren · Pull Request #331 ·
> >>>> > > > apache/flink-statefun (github.com)
> >>>> > > > <https://github.com/apache/flink-statefun/pull/331>.
> >>>> > > >
> >>>> > > > JIRA is here: [FLINK-31619] Upgrade Stateful Functions to Flink
> >>>> 1.16.1
> >>>> > -
> >>>> > > > ASF JIRA (apache.org)
> >>>> > > > <https://issues.apache.org/jira/browse/FLINK-31619?filter=-1>.
> >>>> > > >
> >>>> > > > Statefun references 1.16.2, despite the title -- that version has
> >>>> come
> >>>> > > out
> >>>> > > > since the issue was created.
> >>>> > > >
> >>>> > > > I figured out how to run all the playground tests locally, but it
> >>>> took
> >>>> > a
> >>>> > > > bit of reworking of the playground setup with respect to Docker;
> >>>> > > > specifically, the Docker contexts used to build the example
> >>>> functions
> >>>> > > > needed to be broadened (i.e. moved up the tree) so that, if
> >>>> needed,
> >>>> > local
> >>>> > > > artifacts/code can be accessed from within the containers at build
> >>>> > time.
> >>>> > > > Then I made the Docker compose.yml configurable through
> >>>> environment
> >>>> > > > variables to allow for them to run in either the original manner
> >>>> --
> >>>> > i.e.
> >>>> > > > pulling artifacts from public repos -- or in a "local" mode, where
> >>>> > > > artifacts are pulled from local builds.
> >>>> > > >
> >>>> > > > This process is a cleaner if the playground is a subfolder of the
> >>>> > > > flink-statefun project rather than be its own repository
> >>>> > > > (flink-statefun-playground), because then all the relative paths
> >>>> > between
> >>>> > > > the playground files and the build artifacts are fixed. So, I'd
> >>>> like to
> >>>> > > > propose to move the playground files, modified as described
> >>>> above, to
> >>>> > > > flink-statefun/playground and retire flink-statefun-playground. I
> >>>> can
> >>>> > > > submit separate PR s those changes if everyone is on board.
> >>>> > > >
> >>>> > > > Also, should I plan to do the same upgrade to handle Flink
> >>>> 1.17.x? It
> >>>> > > > should be easy to do, especially while the 1.16.x upgrade is
> >>>> fresh on
> >>>> > my
> >>>> > > > mind.
> >>>> > > >
> >>>> > > > Thanks.
> >>>> > > >
> >>>> > > >
> >>>> > > > On Fri, Aug 11, 2023 at 6:40 PM Galen Warren <
> >>>> galen@cvillewarrens.com>
> >>>> > > > wrote:
> >>>> > > >
> >>>> > > >> I'm done with the code to make Statefun compatible with Flink
> >>>> 1.16,
> >>>> > and
> >>>> > > >> all the tests (including e2e succeed). The required changes were
> >>>> > pretty
> >>>> > > >> minimal.
> >>>> > > >>
> >>>> > > >> I'm running into a bit of a chicken/egg problem executing the
> >>>> tests in
> >>>> > > >> flink-statefun-playground
> >>>> > > >> <https://github.com/apache/flink-statefun-playground>, though.
> >>>> That
> >>>> > > >> project seems to assume that all the various Statefun artifacts
> >>>> are
> >>>> > > built
> >>>> > > >> and deployed to the various public repositories already. I've
> >>>> looked
> >>>> > > into
> >>>> > > >> trying to redirect references to local artifacts; however,
> >>>> that's also
> >>>> > > >> tricky since all the building occurs in Docker containers.
> >>>> > > >>
> >>>> > > >> Gordon, is there a trick to running the sample code in
> >>>> > > >> flink-statefun-playground against yet-unreleased code that I'm
> >>>> > missing?
> >>>> > > >>
> >>>> > > >> Thanks.
> >>>> > > >>
> >>>> > > >> On Sat, Jun 24, 2023 at 12:40 PM Galen Warren <
> >>>> > galen@cvillewarrens.com>
> >>>> > > >> wrote:
> >>>> > > >>
> >>>> > > >>> Great -- thanks!
> >>>> > > >>>
> >>>> > > >>> I'm going to be out of town for about a week but I'll take a
> >>>> look at
> >>>> > > >>> this when I'm back.
> >>>> > > >>>
> >>>> > > >>> On Tue, Jun 20, 2023 at 8:46 AM Martijn Visser <
> >>>> mvisser@confluent.io
> >>>> > >
> >>>> > > >>> wrote:
> >>>> > > >>>
> >>>> > > >>>> Hi Galen,
> >>>> > > >>>>
> >>>> > > >>>> Yes, I'll be more than happy to help with Statefun releases.
> >>>> > > >>>>
> >>>> > > >>>> Best regards,
> >>>> > > >>>>
> >>>> > > >>>> Martijn
> >>>> > > >>>>
> >>>> > > >>>> On Tue, Jun 20, 2023 at 2:21 PM Galen Warren <
> >>>> > galen@cvillewarrens.com
> >>>> > > >
> >>>> > > >>>> wrote:
> >>>> > > >>>>
> >>>> > > >>>>> Thanks.
> >>>> > > >>>>>
> >>>> > > >>>>> Martijn, to answer your question, I'd need to do a small
> >>>> amount of
> >>>> > > >>>>> work to get a PR ready, but not much. Happy to do it if we're
> >>>> > > deciding to
> >>>> > > >>>>> restart Statefun releases -- are we?
> >>>> > > >>>>>
> >>>> > > >>>>> -- Galen
> >>>> > > >>>>>
> >>>> > > >>>>> On Sat, Jun 17, 2023 at 9:47 AM Tzu-Li (Gordon) Tai <
> >>>> > > >>>>> tzulitai@apache.org> wrote:
> >>>> > > >>>>>
> >>>> > > >>>>>> > Perhaps he could weigh in on whether the combination of
> >>>> > automated
> >>>> > > >>>>>> tests plus those smoke tests should be sufficient for
> >>>> testing with
> >>>> > > new
> >>>> > > >>>>>> Flink versions
> >>>> > > >>>>>>
> >>>> > > >>>>>> What we usually did at the bare minimum for new StateFun
> >>>> releases
> >>>> > > was
> >>>> > > >>>>>> the following:
> >>>> > > >>>>>>
> >>>> > > >>>>>>    1. Build tests (including the smoke tests in the e2e
> >>>> module,
> >>>> > > >>>>>>    which covers important tests like exactly-once
> >>>> verification)
> >>>> > > >>>>>>    2. Updating the flink-statefun-playground repo and
> >>>> manually
> >>>> > > >>>>>>    running all language examples there.
> >>>> > > >>>>>>
> >>>> > > >>>>>> If upgrading Flink versions was the only change in the
> >>>> release,
> >>>> > I'd
> >>>> > > >>>>>> probably say that this is sufficient.
> >>>> > > >>>>>>
> >>>> > > >>>>>> Best,
> >>>> > > >>>>>> Gordon
> >>>> > > >>>>>>
> >>>> > > >>>>>> On Thu, Jun 15, 2023 at 5:25 AM Martijn Visser <
> >>>> > > >>>>>> martijnvisser@apache.org> wrote:
> >>>> > > >>>>>>
> >>>> > > >>>>>>> Let me know if you have a PR for a Flink update :)
> >>>> > > >>>>>>>
> >>>> > > >>>>>>> On Thu, Jun 8, 2023 at 5:52 PM Galen Warren via user <
> >>>> > > >>>>>>> user@flink.apache.org> wrote:
> >>>> > > >>>>>>>
> >>>> > > >>>>>>>> Thanks Martijn.
> >>>> > > >>>>>>>>
> >>>> > > >>>>>>>> Personally, I'm already using a local fork of Statefun
> >>>> that is
> >>>> > > >>>>>>>> compatible with Flink 1.16.x, so I wouldn't have any need
> >>>> for a
> >>>> > > released
> >>>> > > >>>>>>>> version compatible with 1.15.x. I'd be happy to do the PRs
> >>>> to
> >>>> > > modify
> >>>> > > >>>>>>>> Statefun to work with new versions of Flink as they come
> >>>> along.
> >>>> > > >>>>>>>>
> >>>> > > >>>>>>>> As for testing, Statefun does have unit tests and Gordon
> >>>> also
> >>>> > sent
> >>>> > > >>>>>>>> me instructions a while back for how to do some additional
> >>>> smoke
> >>>> > > tests
> >>>> > > >>>>>>>> which are pretty straightforward. Perhaps he could weigh
> >>>> in on
> >>>> > > whether the
> >>>> > > >>>>>>>> combination of automated tests plus those smoke tests
> >>>> should be
> >>>> > > sufficient
> >>>> > > >>>>>>>> for testing with new Flink versions (I believe the answer
> >>>> is
> >>>> > yes).
> >>>> > > >>>>>>>>
> >>>> > > >>>>>>>> -- Galen
> >>>> > > >>>>>>>>
> >>>> > > >>>>>>>>
> >>>> > > >>>>>>>>
> >>>> > > >>>>>>>> On Thu, Jun 8, 2023 at 8:01 AM Martijn Visser <
> >>>> > > >>>>>>>> martijnvisser@apache.org> wrote:
> >>>> > > >>>>>>>>
> >>>> > > >>>>>>>>> Hi all,
> >>>> > > >>>>>>>>>
> >>>> > > >>>>>>>>> Apologies for the late reply.
> >>>> > > >>>>>>>>>
> >>>> > > >>>>>>>>> I'm willing to help out with merging requests in Statefun
> >>>> to
> >>>> > keep
> >>>> > > >>>>>>>>> them
> >>>> > > >>>>>>>>> compatible with new Flink releases and create new
> >>>> releases. I
> >>>> > do
> >>>> > > >>>>>>>>> think that
> >>>> > > >>>>>>>>> validation of the functionality of these releases depends
> >>>> a lot
> >>>> > > on
> >>>> > > >>>>>>>>> those
> >>>> > > >>>>>>>>> who do these compatibility updates, with PMC members
> >>>> helping
> >>>> > out
> >>>> > > >>>>>>>>> with the
> >>>> > > >>>>>>>>> formal process.
> >>>> > > >>>>>>>>>
> >>>> > > >>>>>>>>> > Why can't the Apache Software Foundation allow community
> >>>> > > members
> >>>> > > >>>>>>>>> to bring
> >>>> > > >>>>>>>>> it up to date?
> >>>> > > >>>>>>>>>
> >>>> > > >>>>>>>>> There's nothing preventing anyone from reviewing any of
> >>>> the
> >>>> > > >>>>>>>>> current PRs or
> >>>> > > >>>>>>>>> opening new ones. However, none of them are approved [1],
> >>>> so
> >>>> > > >>>>>>>>> there's also
> >>>> > > >>>>>>>>> nothing to merge.
> >>>> > > >>>>>>>>>
> >>>> > > >>>>>>>>> > I believe that there are people and companies on this
> >>>> mailing
> >>>> > > >>>>>>>>> list
> >>>> > > >>>>>>>>> interested in supporting Apache Flink Stateful Functions.
> >>>> > > >>>>>>>>>
> >>>> > > >>>>>>>>> If so, then now is the time to show.
> >>>> > > >>>>>>>>>
> >>>> > > >>>>>>>>> Would there be a preference to create a release with
> >>>> Galen's
> >>>> > > merged
> >>>> > > >>>>>>>>> compatibility update to Flink 1.15.2, or do we want to
> >>>> skip
> >>>> > that
> >>>> > > >>>>>>>>> and go
> >>>> > > >>>>>>>>> straight to a newer version?
> >>>> > > >>>>>>>>>
> >>>> > > >>>>>>>>> Best regards,
> >>>> > > >>>>>>>>>
> >>>> > > >>>>>>>>> Martijn
> >>>> > > >>>>>>>>>
> >>>> > > >>>>>>>>> [1]
> >>>> > > >>>>>>>>>
> >>>> > > >>>>>>>>>
> >>>> > >
> >>>> >
> >>>> https://github.com/apache/flink-statefun/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved
> >>>> > > >>>>>>>>>
> >>>> > > >>>>>>>>> On Tue, Jun 6, 2023 at 3:55 PM Marco Villalobos <
> >>>> > > >>>>>>>>> mvillalobos@kineteque.com>
> >>>> > > >>>>>>>>> wrote:
> >>>> > > >>>>>>>>>
> >>>> > > >>>>>>>>> > Why can't the Apache Software Foundation allow community
> >>>> > > members
> >>>> > > >>>>>>>>> to bring
> >>>> > > >>>>>>>>> > it up to date?
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > What's the process for that?
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > I believe that there are people and companies on this
> >>>> mailing
> >>>> > > >>>>>>>>> list
> >>>> > > >>>>>>>>> > interested in supporting Apache Flink Stateful
> >>>> Functions.
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > You already had two people on this thread express
> >>>> interest.
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > At the very least, we could keep the library versions
> >>>> up to
> >>>> > > date.
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > There are only a small list of new features that might
> >>>> be
> >>>> > > >>>>>>>>> worthwhile:
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > 1. event time processing
> >>>> > > >>>>>>>>> > 2. state rest api
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <
> >>>> > > chesnay@apache.org>
> >>>> > > >>>>>>>>> wrote:
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > If you were to fork it *and want to redistribute it*
> >>>> then the
> >>>> > > >>>>>>>>> short
> >>>> > > >>>>>>>>> > version is that
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> >    1. you have to adhere to the Apache licensing
> >>>> requirements
> >>>> > > >>>>>>>>> >    2. you have to make it clear that your fork does not
> >>>> > belong
> >>>> > > >>>>>>>>> to the
> >>>> > > >>>>>>>>> >    Apache Flink project. (Trademarks and all that)
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > Neither should be significant hurdles (there should
> >>>> also be
> >>>> > > >>>>>>>>> plenty of
> >>>> > > >>>>>>>>> > online resources regarding 1), and if you do this then
> >>>> you
> >>>> > can
> >>>> > > >>>>>>>>> freely share
> >>>> > > >>>>>>>>> > your fork with others.
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > I've also pinged Martijn to take a look at this thread.
> >>>> > > >>>>>>>>> > To my knowledge the project hasn't decided anything yet.
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > On 27/05/2023 04:05, Galen Warren wrote:
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > Ok, I get it. No interest.
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > If this project is being abandoned, I guess I'll work
> >>>> with my
> >>>> > > >>>>>>>>> own fork. Is
> >>>> > > >>>>>>>>> > there anything I should consider here? Can I share it
> >>>> with
> >>>> > > other
> >>>> > > >>>>>>>>> people who
> >>>> > > >>>>>>>>> > use this project?
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > On Tue, May 16, 2023 at 10:50 AM Galen Warren <
> >>>> > > >>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
> >>>> > > >>>>>>>>> > wrote:
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > Hi Martijn, since you opened this discussion thread, I'm
> >>>> > > curious
> >>>> > > >>>>>>>>> what your
> >>>> > > >>>>>>>>> > thoughts are in light of the responses? Thanks.
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <
> >>>> > > >>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
> >>>> > > >>>>>>>>> > wrote:
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > I use Apache Flink for stream processing, and StateFun
> >>>> as a
> >>>> > > >>>>>>>>> hand-off
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > point for the rest of the application.
> >>>> > > >>>>>>>>> > It serves well as a bridge between a Flink Streaming
> >>>> job and
> >>>> > > >>>>>>>>> > micro-services.
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > This is essentially how I use it as well, and I would
> >>>> also be
> >>>> > > >>>>>>>>> sad to see
> >>>> > > >>>>>>>>> > it sunsetted. It works well; I don't know that there is
> >>>> a lot
> >>>> > > of
> >>>> > > >>>>>>>>> new
> >>>> > > >>>>>>>>> > development required, but if there are no new Statefun
> >>>> > > releases,
> >>>> > > >>>>>>>>> then
> >>>> > > >>>>>>>>> > Statefun can only be used with older Flink versions.
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
> >>>> > > >>>>>>>>> mvillalobos@kineteque.com> wrote:
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > I am currently using Stateful Functions in my
> >>>> application.
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > I use Apache Flink for stream processing, and StateFun
> >>>> as a
> >>>> > > >>>>>>>>> hand-off
> >>>> > > >>>>>>>>> > point for the rest of the application.
> >>>> > > >>>>>>>>> > It serves well as a bridge between a Flink Streaming
> >>>> job and
> >>>> > > >>>>>>>>> > micro-services.
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > I would be disappointed if StateFun was sunsetted.  Its
> >>>> a
> >>>> > good
> >>>> > > >>>>>>>>> idea.
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > If there is anything I can do to help, as a contributor
> >>>> > > perhaps,
> >>>> > > >>>>>>>>> please
> >>>> > > >>>>>>>>> > let me know.
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <
> >>>> > > >>>>>>>>> martijnvisser@apache.org> <ma...@apache.org>
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > wrote:
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > Hi everyone,
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > I want to open a discussion on the status of the
> >>>> Statefun
> >>>> > > >>>>>>>>> Project [1]
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > in Apache Flink. As you might have noticed, there
> >>>> hasn't been
> >>>> > > >>>>>>>>> much
> >>>> > > >>>>>>>>> > development over the past months in the Statefun
> >>>> repository
> >>>> > > [2].
> >>>> > > >>>>>>>>> There is
> >>>> > > >>>>>>>>> > currently a lack of active contributors and committers
> >>>> who
> >>>> > are
> >>>> > > >>>>>>>>> able to help
> >>>> > > >>>>>>>>> > with the maintenance of the project.
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > In order to improve the situation, we need to solve the
> >>>> lack
> >>>> > of
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > committers and the lack of contributors.
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > On the lack of committers:
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > 1. Ideally, there are some of the current Flink
> >>>> committers
> >>>> > who
> >>>> > > >>>>>>>>> have
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > the bandwidth and can help with reviewing PRs and
> >>>> merging
> >>>> > them.
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > 2. If that's not an option, it could be a consideration
> >>>> that
> >>>> > > >>>>>>>>> current
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > committers only approve and review PRs, that are
> >>>> approved by
> >>>> > > >>>>>>>>> those who are
> >>>> > > >>>>>>>>> > willing to contribute to Statefun and if the CI passes
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > On the lack of contributors:
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > 3. Next to having this discussion on the Dev and User
> >>>> mailing
> >>>> > > >>>>>>>>> list, we
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > can also create a blog with a call for new contributors
> >>>> on
> >>>> > the
> >>>> > > >>>>>>>>> Flink
> >>>> > > >>>>>>>>> > project website, send out some tweets on the Flink /
> >>>> Statefun
> >>>> > > >>>>>>>>> twitter
> >>>> > > >>>>>>>>> > accounts, post messages on Slack etc. In that message,
> >>>> we
> >>>> > would
> >>>> > > >>>>>>>>> inform how
> >>>> > > >>>>>>>>> > those that are interested in contributing can start and
> >>>> where
> >>>> > > >>>>>>>>> they could
> >>>> > > >>>>>>>>> > reach out for more information.
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > There's also option 4. where a group of interested
> >>>> people
> >>>> > would
> >>>> > > >>>>>>>>> split
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > Statefun from the Flink project and make it a separate
> >>>> top
> >>>> > > level
> >>>> > > >>>>>>>>> project
> >>>> > > >>>>>>>>> > under the Apache Flink umbrella (similar as recently has
> >>>> > > >>>>>>>>> happened with
> >>>> > > >>>>>>>>> > Flink Table Store, which has become Apache Paimon).
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > If we see no improvements in the coming period, we
> >>>> should
> >>>> > > >>>>>>>>> consider
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > sunsetting Statefun and communicate that clearly to the
> >>>> > users.
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > I'm looking forward to your thoughts.
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > Best regards,
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > Martijn
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > [1]
> >>>> > > >>>>>>>>>
> >>>> https://nightlies.apache.org/flink/flink-statefun-docs-master/
> >>>> > <
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> >
> >>>> > https://nightlies.apache.org/flink/flink-statefun-docs-master/
> >>>> > > >
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > [2] https://github.com/apache/flink-statefun <
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > https://github.com/apache/flink-statefun>
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>>
> >>>> > > >>>>>>>>
> >>>> > >
> >>>> >
> >>>>
> >>>

Re: [DISCUSS] Status of Statefun Project

[Martijn Visser <ma...@apache.org>]

Hi everyone,

I'm bringing back this very old thread. It's been a while since we
last discussed this, so I wanted to follow up on where things stand
with Stateful Functions.

 Since the 3.3.0 release in September 2023, there has been:
 - Zero commits to the repository
 - No new releases (now 4+ Flink versions behind: 1.16 vs 1.20/2.2)
 - 22 open PRs unreviewed
 - No sustained contributor activity

I don't think that the current Flink committers and PMC members have
the bandwidth or expertise in StateFun's codebase to maintain it.
StateFun was always a specialized subproject maintained by a small
group of dedicated contributors who have since moved on to other
projects.

Given this situation, I'd like to get the community's input:

1. Is anyone still actively using StateFun in production?
2. Should we consider sunsetting the project rather than leaving it in limbo?

Thanks,

Martijn

Op wo 23 aug 2023 om 08:48 schreef Filip Karnicki <fi...@gmail.com>:
>
> Hi Gordon
>
> Any chance we could get this reviewed and released to the central repo?
> We're currently forced to use a Flink version that has a nasty bug causing
> an operational nightmare
>
> Many thanks
> Fil
>
> On Sat, 19 Aug 2023 at 01:38, Galen Warren via user <us...@flink.apache.org>
> wrote:
>
> > Gotcha, makes sense as to the original division.
> >
> > >> Can this be solved by simply passing in the path to the artifacts
> >
> > This definitely works if we're going to be copying the artifacts on the
> > host side -- into the build context -- and then from the context into the
> > image. It only gets tricky to have a potentially varying path to the
> > artifacts if we're trying to *directly *include the artifacts in the
> > Docker context -- then we have a situation where the Docker context must
> > contain both the artifacts and playground files, with (potentially)
> > different root locations.
> >
> > Maybe the simplest thing to do here is just to leave the playground as-is
> > and then copy the artifacts into the Docker context manually, prior to
> > building the playground images. I'm fine with that. It will mean that each
> > Statefun release will require two PRs and two sets of build/publish steps
> > instead of one, but if everyone else is fine with that I am, too. Unless
> > anyone objects, I'll go ahead and queue up a PR for the playground that
> > makes these changes.
> >
> > Also, I should mention -- in case it's not clear -- that I have already
> > built and run the playground examples with the code from the PR and
> > everything worked. So that PR is ready to move forward with review, etc.,
> > at this point.
> >
> > Thanks.
> >
> >
> >
> >
> >
> >
> >
> > On Fri, Aug 18, 2023 at 4:16 PM Tzu-Li (Gordon) Tai <tz...@apache.org>
> > wrote:
> >
> >> Hi Galen,
> >>
> >> The original intent of having a separate repo for the playground repo,
> >> was that StateFun users can just go to that and start running simple
> >> examples without any other distractions from the core code. I personally
> >> don't have a strong preference here and can understand how it would make
> >> the workflow more streamlined, but just FYI on the reasoning why are
> >> separate in the first place.
> >>
> >> re: paths for locating StateFun artifacts.
> >> Can this be solved by simply passing in the path to the artifacts? As
> >> well as the image tag for the locally build base StateFun image. They could
> >> probably be environment variables.
> >>
> >> Cheers,
> >> Gordon
> >>
> >> On Fri, Aug 18, 2023 at 12:13 PM Galen Warren via user <
> >> user@flink.apache.org> wrote:
> >>
> >>> Yes, exactly! And in addition to the base Statefun jars and the jar for
> >>> the Java SDK, it does an equivalent copy/register operation for each of the
> >>> other SDK libraries (Go, Python, Javascript) so that those libraries are
> >>> also available when building the playground examples.
> >>>
> >>> One more question: In order to copy the various build artifacts into the
> >>> Docker containers, those artifacts need to be part of the Docker context.
> >>> With the playground being a separate project, that's slightly tricky to do,
> >>> as there is no guarantee (other than convention) about the relative paths
> >>> of *flink-statefun* and* flink-statefun-playground *in someone's local
> >>> filesystem. The way I've set this up locally is to copy the playground into
> >>> the* flink-statefun* project -- i.e. to *flink-statefun*/playground --
> >>> such that I can set the Docker context to the root folder of
> >>> *flink-statefun* and then have access to any local code and/or build
> >>> artifacts.
> >>>
> >>> I'm wondering if there might be any appetite for making that move
> >>> permanent, i.e. moving the playground to *flink-statefun*/playground
> >>> and deprecating the standalone playground project. In addition to making
> >>> the problem of building with unreleased artifacts a bit simpler to solve,
> >>> it would also simplify the process of releasing a new Statefun version,
> >>> since the entire process could be handled with a single PR and associated
> >>> build/deploy tasks. In other words, a single PR could both update and
> >>> deploy the Statefun package and the playground code and images.
> >>>
> >>> As it stands, at least two PRs would be required for each Statefun
> >>> version update -- one for *flink-statefun* and one for
> >>> *flink-statefun-playground*.
> >>>
> >>> Anyway, just an idea. Maybe there's an important reason for these
> >>> projects to remain separate. If we do want to keep the playground project
> >>> where it is, I could solve the copying problem by requiring the two
> >>> projects to be siblings in the file system and by pre-copying the local
> >>> build artifacts into a location accessible by the existing Docker contexts.
> >>> This would still leave us with the need to have two PRs and releases
> >>> instead of one, though.
> >>>
> >>> Thanks for your help!
> >>>
> >>>
> >>> On Fri, Aug 18, 2023 at 2:45 PM Tzu-Li (Gordon) Tai <tz...@apache.org>
> >>> wrote:
> >>>
> >>>> Hi Galen,
> >>>>
> >>>> > locally built code is copied into the build containers
> >>>> so that it can be accessed during the build.
> >>>>
> >>>> That's exactly what we had been doing for release testing, yes. Sorry I
> >>>> missed that detail in my previous response.
> >>>>
> >>>> And yes, that sounds like a reasonable approach. If I understand you
> >>>> correctly, the workflow would become this:
> >>>>
> >>>>    1. Build the StateFun repo locally to install the snapshot artifact
> >>>>    jars + have a local base StateFun image.
> >>>>    2. Run the playground in "local" mode, so that it uses the local base
> >>>>    StateFun image + builds the playground code using copied artifact
> >>>> jars
> >>>>    (instead of pulling from Maven).
> >>>>
> >>>> That looks good to me!
> >>>>
> >>>> Thanks,
> >>>> Gordon
> >>>>
> >>>> On Fri, Aug 18, 2023 at 11:33 AM Galen Warren
> >>>> <ga...@cvillewarrens.com.invalid> wrote:
> >>>>
> >>>> > Thanks.
> >>>> >
> >>>> > If you were to build a local image, as you suggest, how do you access
> >>>> that
> >>>> > image when building the playground images? All the compilation of
> >>>> > playground code happens inside containers, so local images on the host
> >>>> > aren't available in those containers. Unless I'm missing something?
> >>>> >
> >>>> > I've slightly reworked things such that the playground images can be
> >>>> run in
> >>>> > one of two modes -- the default mode, which works like before, and a
> >>>> > "local" mode where locally built code is copied into the build
> >>>> containers
> >>>> > so that it can be accessed during the build. It works fine, you just
> >>>> have
> >>>> > to define a couple of environment variables when running
> >>>> docker-compose to
> >>>> > specify default vs. local mode and what versions of Flink and Statefun
> >>>> > should be referenced, and then you can build a run the local examples
> >>>> > without any additional steps. Does that sound like a reasonable
> >>>> approach?
> >>>> >
> >>>> >
> >>>> > On Fri, Aug 18, 2023 at 2:17 PM Tzu-Li (Gordon) Tai <
> >>>> tzulitai@apache.org>
> >>>> > wrote:
> >>>> >
> >>>> > > Hi Galen,
> >>>> > >
> >>>> > > > Gordon, is there a trick to running the sample code in
> >>>> > > flink-statefun-playground against yet-unreleased code that I'm
> >>>> missing?
> >>>> > >
> >>>> > > You'd have to locally build an image from the release branch, with a
> >>>> > > temporary image version tag. Then, in the flink-statefun-playground,
> >>>> > change
> >>>> > > the image versions in the docker-compose files to use that locally
> >>>> built
> >>>> > > image. IIRC, that's what we have been doing in the past.
> >>>> Admittedly, it's
> >>>> > > pretty manual - I don't think the CI manages this workflow.
> >>>> > >
> >>>> > > Thanks,
> >>>> > > Gordon
> >>>> > >
> >>>> > > On Mon, Aug 14, 2023 at 10:42 AM Galen Warren <
> >>>> galen@cvillewarrens.com>
> >>>> > > wrote:
> >>>> > >
> >>>> > > > I created a pull request for this: [FLINK-31619] Upgrade Stateful
> >>>> > > > Functions to Flink 1.16.1 by galenwarren · Pull Request #331 ·
> >>>> > > > apache/flink-statefun (github.com)
> >>>> > > > <https://github.com/apache/flink-statefun/pull/331>.
> >>>> > > >
> >>>> > > > JIRA is here: [FLINK-31619] Upgrade Stateful Functions to Flink
> >>>> 1.16.1
> >>>> > -
> >>>> > > > ASF JIRA (apache.org)
> >>>> > > > <https://issues.apache.org/jira/browse/FLINK-31619?filter=-1>.
> >>>> > > >
> >>>> > > > Statefun references 1.16.2, despite the title -- that version has
> >>>> come
> >>>> > > out
> >>>> > > > since the issue was created.
> >>>> > > >
> >>>> > > > I figured out how to run all the playground tests locally, but it
> >>>> took
> >>>> > a
> >>>> > > > bit of reworking of the playground setup with respect to Docker;
> >>>> > > > specifically, the Docker contexts used to build the example
> >>>> functions
> >>>> > > > needed to be broadened (i.e. moved up the tree) so that, if
> >>>> needed,
> >>>> > local
> >>>> > > > artifacts/code can be accessed from within the containers at build
> >>>> > time.
> >>>> > > > Then I made the Docker compose.yml configurable through
> >>>> environment
> >>>> > > > variables to allow for them to run in either the original manner
> >>>> --
> >>>> > i.e.
> >>>> > > > pulling artifacts from public repos -- or in a "local" mode, where
> >>>> > > > artifacts are pulled from local builds.
> >>>> > > >
> >>>> > > > This process is a cleaner if the playground is a subfolder of the
> >>>> > > > flink-statefun project rather than be its own repository
> >>>> > > > (flink-statefun-playground), because then all the relative paths
> >>>> > between
> >>>> > > > the playground files and the build artifacts are fixed. So, I'd
> >>>> like to
> >>>> > > > propose to move the playground files, modified as described
> >>>> above, to
> >>>> > > > flink-statefun/playground and retire flink-statefun-playground. I
> >>>> can
> >>>> > > > submit separate PR s those changes if everyone is on board.
> >>>> > > >
> >>>> > > > Also, should I plan to do the same upgrade to handle Flink
> >>>> 1.17.x? It
> >>>> > > > should be easy to do, especially while the 1.16.x upgrade is
> >>>> fresh on
> >>>> > my
> >>>> > > > mind.
> >>>> > > >
> >>>> > > > Thanks.
> >>>> > > >
> >>>> > > >
> >>>> > > > On Fri, Aug 11, 2023 at 6:40 PM Galen Warren <
> >>>> galen@cvillewarrens.com>
> >>>> > > > wrote:
> >>>> > > >
> >>>> > > >> I'm done with the code to make Statefun compatible with Flink
> >>>> 1.16,
> >>>> > and
> >>>> > > >> all the tests (including e2e succeed). The required changes were
> >>>> > pretty
> >>>> > > >> minimal.
> >>>> > > >>
> >>>> > > >> I'm running into a bit of a chicken/egg problem executing the
> >>>> tests in
> >>>> > > >> flink-statefun-playground
> >>>> > > >> <https://github.com/apache/flink-statefun-playground>, though.
> >>>> That
> >>>> > > >> project seems to assume that all the various Statefun artifacts
> >>>> are
> >>>> > > built
> >>>> > > >> and deployed to the various public repositories already. I've
> >>>> looked
> >>>> > > into
> >>>> > > >> trying to redirect references to local artifacts; however,
> >>>> that's also
> >>>> > > >> tricky since all the building occurs in Docker containers.
> >>>> > > >>
> >>>> > > >> Gordon, is there a trick to running the sample code in
> >>>> > > >> flink-statefun-playground against yet-unreleased code that I'm
> >>>> > missing?
> >>>> > > >>
> >>>> > > >> Thanks.
> >>>> > > >>
> >>>> > > >> On Sat, Jun 24, 2023 at 12:40 PM Galen Warren <
> >>>> > galen@cvillewarrens.com>
> >>>> > > >> wrote:
> >>>> > > >>
> >>>> > > >>> Great -- thanks!
> >>>> > > >>>
> >>>> > > >>> I'm going to be out of town for about a week but I'll take a
> >>>> look at
> >>>> > > >>> this when I'm back.
> >>>> > > >>>
> >>>> > > >>> On Tue, Jun 20, 2023 at 8:46 AM Martijn Visser <
> >>>> mvisser@confluent.io
> >>>> > >
> >>>> > > >>> wrote:
> >>>> > > >>>
> >>>> > > >>>> Hi Galen,
> >>>> > > >>>>
> >>>> > > >>>> Yes, I'll be more than happy to help with Statefun releases.
> >>>> > > >>>>
> >>>> > > >>>> Best regards,
> >>>> > > >>>>
> >>>> > > >>>> Martijn
> >>>> > > >>>>
> >>>> > > >>>> On Tue, Jun 20, 2023 at 2:21 PM Galen Warren <
> >>>> > galen@cvillewarrens.com
> >>>> > > >
> >>>> > > >>>> wrote:
> >>>> > > >>>>
> >>>> > > >>>>> Thanks.
> >>>> > > >>>>>
> >>>> > > >>>>> Martijn, to answer your question, I'd need to do a small
> >>>> amount of
> >>>> > > >>>>> work to get a PR ready, but not much. Happy to do it if we're
> >>>> > > deciding to
> >>>> > > >>>>> restart Statefun releases -- are we?
> >>>> > > >>>>>
> >>>> > > >>>>> -- Galen
> >>>> > > >>>>>
> >>>> > > >>>>> On Sat, Jun 17, 2023 at 9:47 AM Tzu-Li (Gordon) Tai <
> >>>> > > >>>>> tzulitai@apache.org> wrote:
> >>>> > > >>>>>
> >>>> > > >>>>>> > Perhaps he could weigh in on whether the combination of
> >>>> > automated
> >>>> > > >>>>>> tests plus those smoke tests should be sufficient for
> >>>> testing with
> >>>> > > new
> >>>> > > >>>>>> Flink versions
> >>>> > > >>>>>>
> >>>> > > >>>>>> What we usually did at the bare minimum for new StateFun
> >>>> releases
> >>>> > > was
> >>>> > > >>>>>> the following:
> >>>> > > >>>>>>
> >>>> > > >>>>>>    1. Build tests (including the smoke tests in the e2e
> >>>> module,
> >>>> > > >>>>>>    which covers important tests like exactly-once
> >>>> verification)
> >>>> > > >>>>>>    2. Updating the flink-statefun-playground repo and
> >>>> manually
> >>>> > > >>>>>>    running all language examples there.
> >>>> > > >>>>>>
> >>>> > > >>>>>> If upgrading Flink versions was the only change in the
> >>>> release,
> >>>> > I'd
> >>>> > > >>>>>> probably say that this is sufficient.
> >>>> > > >>>>>>
> >>>> > > >>>>>> Best,
> >>>> > > >>>>>> Gordon
> >>>> > > >>>>>>
> >>>> > > >>>>>> On Thu, Jun 15, 2023 at 5:25 AM Martijn Visser <
> >>>> > > >>>>>> martijnvisser@apache.org> wrote:
> >>>> > > >>>>>>
> >>>> > > >>>>>>> Let me know if you have a PR for a Flink update :)
> >>>> > > >>>>>>>
> >>>> > > >>>>>>> On Thu, Jun 8, 2023 at 5:52 PM Galen Warren via user <
> >>>> > > >>>>>>> user@flink.apache.org> wrote:
> >>>> > > >>>>>>>
> >>>> > > >>>>>>>> Thanks Martijn.
> >>>> > > >>>>>>>>
> >>>> > > >>>>>>>> Personally, I'm already using a local fork of Statefun
> >>>> that is
> >>>> > > >>>>>>>> compatible with Flink 1.16.x, so I wouldn't have any need
> >>>> for a
> >>>> > > released
> >>>> > > >>>>>>>> version compatible with 1.15.x. I'd be happy to do the PRs
> >>>> to
> >>>> > > modify
> >>>> > > >>>>>>>> Statefun to work with new versions of Flink as they come
> >>>> along.
> >>>> > > >>>>>>>>
> >>>> > > >>>>>>>> As for testing, Statefun does have unit tests and Gordon
> >>>> also
> >>>> > sent
> >>>> > > >>>>>>>> me instructions a while back for how to do some additional
> >>>> smoke
> >>>> > > tests
> >>>> > > >>>>>>>> which are pretty straightforward. Perhaps he could weigh
> >>>> in on
> >>>> > > whether the
> >>>> > > >>>>>>>> combination of automated tests plus those smoke tests
> >>>> should be
> >>>> > > sufficient
> >>>> > > >>>>>>>> for testing with new Flink versions (I believe the answer
> >>>> is
> >>>> > yes).
> >>>> > > >>>>>>>>
> >>>> > > >>>>>>>> -- Galen
> >>>> > > >>>>>>>>
> >>>> > > >>>>>>>>
> >>>> > > >>>>>>>>
> >>>> > > >>>>>>>> On Thu, Jun 8, 2023 at 8:01 AM Martijn Visser <
> >>>> > > >>>>>>>> martijnvisser@apache.org> wrote:
> >>>> > > >>>>>>>>
> >>>> > > >>>>>>>>> Hi all,
> >>>> > > >>>>>>>>>
> >>>> > > >>>>>>>>> Apologies for the late reply.
> >>>> > > >>>>>>>>>
> >>>> > > >>>>>>>>> I'm willing to help out with merging requests in Statefun
> >>>> to
> >>>> > keep
> >>>> > > >>>>>>>>> them
> >>>> > > >>>>>>>>> compatible with new Flink releases and create new
> >>>> releases. I
> >>>> > do
> >>>> > > >>>>>>>>> think that
> >>>> > > >>>>>>>>> validation of the functionality of these releases depends
> >>>> a lot
> >>>> > > on
> >>>> > > >>>>>>>>> those
> >>>> > > >>>>>>>>> who do these compatibility updates, with PMC members
> >>>> helping
> >>>> > out
> >>>> > > >>>>>>>>> with the
> >>>> > > >>>>>>>>> formal process.
> >>>> > > >>>>>>>>>
> >>>> > > >>>>>>>>> > Why can't the Apache Software Foundation allow community
> >>>> > > members
> >>>> > > >>>>>>>>> to bring
> >>>> > > >>>>>>>>> it up to date?
> >>>> > > >>>>>>>>>
> >>>> > > >>>>>>>>> There's nothing preventing anyone from reviewing any of
> >>>> the
> >>>> > > >>>>>>>>> current PRs or
> >>>> > > >>>>>>>>> opening new ones. However, none of them are approved [1],
> >>>> so
> >>>> > > >>>>>>>>> there's also
> >>>> > > >>>>>>>>> nothing to merge.
> >>>> > > >>>>>>>>>
> >>>> > > >>>>>>>>> > I believe that there are people and companies on this
> >>>> mailing
> >>>> > > >>>>>>>>> list
> >>>> > > >>>>>>>>> interested in supporting Apache Flink Stateful Functions.
> >>>> > > >>>>>>>>>
> >>>> > > >>>>>>>>> If so, then now is the time to show.
> >>>> > > >>>>>>>>>
> >>>> > > >>>>>>>>> Would there be a preference to create a release with
> >>>> Galen's
> >>>> > > merged
> >>>> > > >>>>>>>>> compatibility update to Flink 1.15.2, or do we want to
> >>>> skip
> >>>> > that
> >>>> > > >>>>>>>>> and go
> >>>> > > >>>>>>>>> straight to a newer version?
> >>>> > > >>>>>>>>>
> >>>> > > >>>>>>>>> Best regards,
> >>>> > > >>>>>>>>>
> >>>> > > >>>>>>>>> Martijn
> >>>> > > >>>>>>>>>
> >>>> > > >>>>>>>>> [1]
> >>>> > > >>>>>>>>>
> >>>> > > >>>>>>>>>
> >>>> > >
> >>>> >
> >>>> https://github.com/apache/flink-statefun/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved
> >>>> > > >>>>>>>>>
> >>>> > > >>>>>>>>> On Tue, Jun 6, 2023 at 3:55 PM Marco Villalobos <
> >>>> > > >>>>>>>>> mvillalobos@kineteque.com>
> >>>> > > >>>>>>>>> wrote:
> >>>> > > >>>>>>>>>
> >>>> > > >>>>>>>>> > Why can't the Apache Software Foundation allow community
> >>>> > > members
> >>>> > > >>>>>>>>> to bring
> >>>> > > >>>>>>>>> > it up to date?
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > What's the process for that?
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > I believe that there are people and companies on this
> >>>> mailing
> >>>> > > >>>>>>>>> list
> >>>> > > >>>>>>>>> > interested in supporting Apache Flink Stateful
> >>>> Functions.
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > You already had two people on this thread express
> >>>> interest.
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > At the very least, we could keep the library versions
> >>>> up to
> >>>> > > date.
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > There are only a small list of new features that might
> >>>> be
> >>>> > > >>>>>>>>> worthwhile:
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > 1. event time processing
> >>>> > > >>>>>>>>> > 2. state rest api
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <
> >>>> > > chesnay@apache.org>
> >>>> > > >>>>>>>>> wrote:
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > If you were to fork it *and want to redistribute it*
> >>>> then the
> >>>> > > >>>>>>>>> short
> >>>> > > >>>>>>>>> > version is that
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> >    1. you have to adhere to the Apache licensing
> >>>> requirements
> >>>> > > >>>>>>>>> >    2. you have to make it clear that your fork does not
> >>>> > belong
> >>>> > > >>>>>>>>> to the
> >>>> > > >>>>>>>>> >    Apache Flink project. (Trademarks and all that)
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > Neither should be significant hurdles (there should
> >>>> also be
> >>>> > > >>>>>>>>> plenty of
> >>>> > > >>>>>>>>> > online resources regarding 1), and if you do this then
> >>>> you
> >>>> > can
> >>>> > > >>>>>>>>> freely share
> >>>> > > >>>>>>>>> > your fork with others.
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > I've also pinged Martijn to take a look at this thread.
> >>>> > > >>>>>>>>> > To my knowledge the project hasn't decided anything yet.
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > On 27/05/2023 04:05, Galen Warren wrote:
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > Ok, I get it. No interest.
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > If this project is being abandoned, I guess I'll work
> >>>> with my
> >>>> > > >>>>>>>>> own fork. Is
> >>>> > > >>>>>>>>> > there anything I should consider here? Can I share it
> >>>> with
> >>>> > > other
> >>>> > > >>>>>>>>> people who
> >>>> > > >>>>>>>>> > use this project?
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > On Tue, May 16, 2023 at 10:50 AM Galen Warren <
> >>>> > > >>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
> >>>> > > >>>>>>>>> > wrote:
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > Hi Martijn, since you opened this discussion thread, I'm
> >>>> > > curious
> >>>> > > >>>>>>>>> what your
> >>>> > > >>>>>>>>> > thoughts are in light of the responses? Thanks.
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <
> >>>> > > >>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
> >>>> > > >>>>>>>>> > wrote:
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > I use Apache Flink for stream processing, and StateFun
> >>>> as a
> >>>> > > >>>>>>>>> hand-off
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > point for the rest of the application.
> >>>> > > >>>>>>>>> > It serves well as a bridge between a Flink Streaming
> >>>> job and
> >>>> > > >>>>>>>>> > micro-services.
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > This is essentially how I use it as well, and I would
> >>>> also be
> >>>> > > >>>>>>>>> sad to see
> >>>> > > >>>>>>>>> > it sunsetted. It works well; I don't know that there is
> >>>> a lot
> >>>> > > of
> >>>> > > >>>>>>>>> new
> >>>> > > >>>>>>>>> > development required, but if there are no new Statefun
> >>>> > > releases,
> >>>> > > >>>>>>>>> then
> >>>> > > >>>>>>>>> > Statefun can only be used with older Flink versions.
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
> >>>> > > >>>>>>>>> mvillalobos@kineteque.com> wrote:
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > I am currently using Stateful Functions in my
> >>>> application.
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > I use Apache Flink for stream processing, and StateFun
> >>>> as a
> >>>> > > >>>>>>>>> hand-off
> >>>> > > >>>>>>>>> > point for the rest of the application.
> >>>> > > >>>>>>>>> > It serves well as a bridge between a Flink Streaming
> >>>> job and
> >>>> > > >>>>>>>>> > micro-services.
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > I would be disappointed if StateFun was sunsetted.  Its
> >>>> a
> >>>> > good
> >>>> > > >>>>>>>>> idea.
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > If there is anything I can do to help, as a contributor
> >>>> > > perhaps,
> >>>> > > >>>>>>>>> please
> >>>> > > >>>>>>>>> > let me know.
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <
> >>>> > > >>>>>>>>> martijnvisser@apache.org> <ma...@apache.org>
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > wrote:
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > Hi everyone,
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > I want to open a discussion on the status of the
> >>>> Statefun
> >>>> > > >>>>>>>>> Project [1]
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > in Apache Flink. As you might have noticed, there
> >>>> hasn't been
> >>>> > > >>>>>>>>> much
> >>>> > > >>>>>>>>> > development over the past months in the Statefun
> >>>> repository
> >>>> > > [2].
> >>>> > > >>>>>>>>> There is
> >>>> > > >>>>>>>>> > currently a lack of active contributors and committers
> >>>> who
> >>>> > are
> >>>> > > >>>>>>>>> able to help
> >>>> > > >>>>>>>>> > with the maintenance of the project.
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > In order to improve the situation, we need to solve the
> >>>> lack
> >>>> > of
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > committers and the lack of contributors.
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > On the lack of committers:
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > 1. Ideally, there are some of the current Flink
> >>>> committers
> >>>> > who
> >>>> > > >>>>>>>>> have
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > the bandwidth and can help with reviewing PRs and
> >>>> merging
> >>>> > them.
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > 2. If that's not an option, it could be a consideration
> >>>> that
> >>>> > > >>>>>>>>> current
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > committers only approve and review PRs, that are
> >>>> approved by
> >>>> > > >>>>>>>>> those who are
> >>>> > > >>>>>>>>> > willing to contribute to Statefun and if the CI passes
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > On the lack of contributors:
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > 3. Next to having this discussion on the Dev and User
> >>>> mailing
> >>>> > > >>>>>>>>> list, we
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > can also create a blog with a call for new contributors
> >>>> on
> >>>> > the
> >>>> > > >>>>>>>>> Flink
> >>>> > > >>>>>>>>> > project website, send out some tweets on the Flink /
> >>>> Statefun
> >>>> > > >>>>>>>>> twitter
> >>>> > > >>>>>>>>> > accounts, post messages on Slack etc. In that message,
> >>>> we
> >>>> > would
> >>>> > > >>>>>>>>> inform how
> >>>> > > >>>>>>>>> > those that are interested in contributing can start and
> >>>> where
> >>>> > > >>>>>>>>> they could
> >>>> > > >>>>>>>>> > reach out for more information.
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > There's also option 4. where a group of interested
> >>>> people
> >>>> > would
> >>>> > > >>>>>>>>> split
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > Statefun from the Flink project and make it a separate
> >>>> top
> >>>> > > level
> >>>> > > >>>>>>>>> project
> >>>> > > >>>>>>>>> > under the Apache Flink umbrella (similar as recently has
> >>>> > > >>>>>>>>> happened with
> >>>> > > >>>>>>>>> > Flink Table Store, which has become Apache Paimon).
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > If we see no improvements in the coming period, we
> >>>> should
> >>>> > > >>>>>>>>> consider
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > sunsetting Statefun and communicate that clearly to the
> >>>> > users.
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > I'm looking forward to your thoughts.
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > Best regards,
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > Martijn
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > [1]
> >>>> > > >>>>>>>>>
> >>>> https://nightlies.apache.org/flink/flink-statefun-docs-master/
> >>>> > <
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> >
> >>>> > https://nightlies.apache.org/flink/flink-statefun-docs-master/
> >>>> > > >
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > [2] https://github.com/apache/flink-statefun <
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> > https://github.com/apache/flink-statefun>
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>> >
> >>>> > > >>>>>>>>>
> >>>> > > >>>>>>>>
> >>>> > >
> >>>> >
> >>>>
> >>>

Re: [DISCUSS] Status of Statefun Project

[Filip Karnicki <fi...@gmail.com>]

Hi Gordon

Any chance we could get this reviewed and released to the central repo?
We're currently forced to use a Flink version that has a nasty bug causing
an operational nightmare

Many thanks
Fil

On Sat, 19 Aug 2023 at 01:38, Galen Warren via user <us...@flink.apache.org>
wrote:

> Gotcha, makes sense as to the original division.
>
> >> Can this be solved by simply passing in the path to the artifacts
>
> This definitely works if we're going to be copying the artifacts on the
> host side -- into the build context -- and then from the context into the
> image. It only gets tricky to have a potentially varying path to the
> artifacts if we're trying to *directly *include the artifacts in the
> Docker context -- then we have a situation where the Docker context must
> contain both the artifacts and playground files, with (potentially)
> different root locations.
>
> Maybe the simplest thing to do here is just to leave the playground as-is
> and then copy the artifacts into the Docker context manually, prior to
> building the playground images. I'm fine with that. It will mean that each
> Statefun release will require two PRs and two sets of build/publish steps
> instead of one, but if everyone else is fine with that I am, too. Unless
> anyone objects, I'll go ahead and queue up a PR for the playground that
> makes these changes.
>
> Also, I should mention -- in case it's not clear -- that I have already
> built and run the playground examples with the code from the PR and
> everything worked. So that PR is ready to move forward with review, etc.,
> at this point.
>
> Thanks.
>
>
>
>
>
>
>
> On Fri, Aug 18, 2023 at 4:16 PM Tzu-Li (Gordon) Tai <tz...@apache.org>
> wrote:
>
>> Hi Galen,
>>
>> The original intent of having a separate repo for the playground repo,
>> was that StateFun users can just go to that and start running simple
>> examples without any other distractions from the core code. I personally
>> don't have a strong preference here and can understand how it would make
>> the workflow more streamlined, but just FYI on the reasoning why are
>> separate in the first place.
>>
>> re: paths for locating StateFun artifacts.
>> Can this be solved by simply passing in the path to the artifacts? As
>> well as the image tag for the locally build base StateFun image. They could
>> probably be environment variables.
>>
>> Cheers,
>> Gordon
>>
>> On Fri, Aug 18, 2023 at 12:13 PM Galen Warren via user <
>> user@flink.apache.org> wrote:
>>
>>> Yes, exactly! And in addition to the base Statefun jars and the jar for
>>> the Java SDK, it does an equivalent copy/register operation for each of the
>>> other SDK libraries (Go, Python, Javascript) so that those libraries are
>>> also available when building the playground examples.
>>>
>>> One more question: In order to copy the various build artifacts into the
>>> Docker containers, those artifacts need to be part of the Docker context.
>>> With the playground being a separate project, that's slightly tricky to do,
>>> as there is no guarantee (other than convention) about the relative paths
>>> of *flink-statefun* and* flink-statefun-playground *in someone's local
>>> filesystem. The way I've set this up locally is to copy the playground into
>>> the* flink-statefun* project -- i.e. to *flink-statefun*/playground --
>>> such that I can set the Docker context to the root folder of
>>> *flink-statefun* and then have access to any local code and/or build
>>> artifacts.
>>>
>>> I'm wondering if there might be any appetite for making that move
>>> permanent, i.e. moving the playground to *flink-statefun*/playground
>>> and deprecating the standalone playground project. In addition to making
>>> the problem of building with unreleased artifacts a bit simpler to solve,
>>> it would also simplify the process of releasing a new Statefun version,
>>> since the entire process could be handled with a single PR and associated
>>> build/deploy tasks. In other words, a single PR could both update and
>>> deploy the Statefun package and the playground code and images.
>>>
>>> As it stands, at least two PRs would be required for each Statefun
>>> version update -- one for *flink-statefun* and one for
>>> *flink-statefun-playground*.
>>>
>>> Anyway, just an idea. Maybe there's an important reason for these
>>> projects to remain separate. If we do want to keep the playground project
>>> where it is, I could solve the copying problem by requiring the two
>>> projects to be siblings in the file system and by pre-copying the local
>>> build artifacts into a location accessible by the existing Docker contexts.
>>> This would still leave us with the need to have two PRs and releases
>>> instead of one, though.
>>>
>>> Thanks for your help!
>>>
>>>
>>> On Fri, Aug 18, 2023 at 2:45 PM Tzu-Li (Gordon) Tai <tz...@apache.org>
>>> wrote:
>>>
>>>> Hi Galen,
>>>>
>>>> > locally built code is copied into the build containers
>>>> so that it can be accessed during the build.
>>>>
>>>> That's exactly what we had been doing for release testing, yes. Sorry I
>>>> missed that detail in my previous response.
>>>>
>>>> And yes, that sounds like a reasonable approach. If I understand you
>>>> correctly, the workflow would become this:
>>>>
>>>>    1. Build the StateFun repo locally to install the snapshot artifact
>>>>    jars + have a local base StateFun image.
>>>>    2. Run the playground in "local" mode, so that it uses the local base
>>>>    StateFun image + builds the playground code using copied artifact
>>>> jars
>>>>    (instead of pulling from Maven).
>>>>
>>>> That looks good to me!
>>>>
>>>> Thanks,
>>>> Gordon
>>>>
>>>> On Fri, Aug 18, 2023 at 11:33 AM Galen Warren
>>>> <ga...@cvillewarrens.com.invalid> wrote:
>>>>
>>>> > Thanks.
>>>> >
>>>> > If you were to build a local image, as you suggest, how do you access
>>>> that
>>>> > image when building the playground images? All the compilation of
>>>> > playground code happens inside containers, so local images on the host
>>>> > aren't available in those containers. Unless I'm missing something?
>>>> >
>>>> > I've slightly reworked things such that the playground images can be
>>>> run in
>>>> > one of two modes -- the default mode, which works like before, and a
>>>> > "local" mode where locally built code is copied into the build
>>>> containers
>>>> > so that it can be accessed during the build. It works fine, you just
>>>> have
>>>> > to define a couple of environment variables when running
>>>> docker-compose to
>>>> > specify default vs. local mode and what versions of Flink and Statefun
>>>> > should be referenced, and then you can build a run the local examples
>>>> > without any additional steps. Does that sound like a reasonable
>>>> approach?
>>>> >
>>>> >
>>>> > On Fri, Aug 18, 2023 at 2:17 PM Tzu-Li (Gordon) Tai <
>>>> tzulitai@apache.org>
>>>> > wrote:
>>>> >
>>>> > > Hi Galen,
>>>> > >
>>>> > > > Gordon, is there a trick to running the sample code in
>>>> > > flink-statefun-playground against yet-unreleased code that I'm
>>>> missing?
>>>> > >
>>>> > > You'd have to locally build an image from the release branch, with a
>>>> > > temporary image version tag. Then, in the flink-statefun-playground,
>>>> > change
>>>> > > the image versions in the docker-compose files to use that locally
>>>> built
>>>> > > image. IIRC, that's what we have been doing in the past.
>>>> Admittedly, it's
>>>> > > pretty manual - I don't think the CI manages this workflow.
>>>> > >
>>>> > > Thanks,
>>>> > > Gordon
>>>> > >
>>>> > > On Mon, Aug 14, 2023 at 10:42 AM Galen Warren <
>>>> galen@cvillewarrens.com>
>>>> > > wrote:
>>>> > >
>>>> > > > I created a pull request for this: [FLINK-31619] Upgrade Stateful
>>>> > > > Functions to Flink 1.16.1 by galenwarren · Pull Request #331 ·
>>>> > > > apache/flink-statefun (github.com)
>>>> > > > <https://github.com/apache/flink-statefun/pull/331>.
>>>> > > >
>>>> > > > JIRA is here: [FLINK-31619] Upgrade Stateful Functions to Flink
>>>> 1.16.1
>>>> > -
>>>> > > > ASF JIRA (apache.org)
>>>> > > > <https://issues.apache.org/jira/browse/FLINK-31619?filter=-1>.
>>>> > > >
>>>> > > > Statefun references 1.16.2, despite the title -- that version has
>>>> come
>>>> > > out
>>>> > > > since the issue was created.
>>>> > > >
>>>> > > > I figured out how to run all the playground tests locally, but it
>>>> took
>>>> > a
>>>> > > > bit of reworking of the playground setup with respect to Docker;
>>>> > > > specifically, the Docker contexts used to build the example
>>>> functions
>>>> > > > needed to be broadened (i.e. moved up the tree) so that, if
>>>> needed,
>>>> > local
>>>> > > > artifacts/code can be accessed from within the containers at build
>>>> > time.
>>>> > > > Then I made the Docker compose.yml configurable through
>>>> environment
>>>> > > > variables to allow for them to run in either the original manner
>>>> --
>>>> > i.e.
>>>> > > > pulling artifacts from public repos -- or in a "local" mode, where
>>>> > > > artifacts are pulled from local builds.
>>>> > > >
>>>> > > > This process is a cleaner if the playground is a subfolder of the
>>>> > > > flink-statefun project rather than be its own repository
>>>> > > > (flink-statefun-playground), because then all the relative paths
>>>> > between
>>>> > > > the playground files and the build artifacts are fixed. So, I'd
>>>> like to
>>>> > > > propose to move the playground files, modified as described
>>>> above, to
>>>> > > > flink-statefun/playground and retire flink-statefun-playground. I
>>>> can
>>>> > > > submit separate PR s those changes if everyone is on board.
>>>> > > >
>>>> > > > Also, should I plan to do the same upgrade to handle Flink
>>>> 1.17.x? It
>>>> > > > should be easy to do, especially while the 1.16.x upgrade is
>>>> fresh on
>>>> > my
>>>> > > > mind.
>>>> > > >
>>>> > > > Thanks.
>>>> > > >
>>>> > > >
>>>> > > > On Fri, Aug 11, 2023 at 6:40 PM Galen Warren <
>>>> galen@cvillewarrens.com>
>>>> > > > wrote:
>>>> > > >
>>>> > > >> I'm done with the code to make Statefun compatible with Flink
>>>> 1.16,
>>>> > and
>>>> > > >> all the tests (including e2e succeed). The required changes were
>>>> > pretty
>>>> > > >> minimal.
>>>> > > >>
>>>> > > >> I'm running into a bit of a chicken/egg problem executing the
>>>> tests in
>>>> > > >> flink-statefun-playground
>>>> > > >> <https://github.com/apache/flink-statefun-playground>, though.
>>>> That
>>>> > > >> project seems to assume that all the various Statefun artifacts
>>>> are
>>>> > > built
>>>> > > >> and deployed to the various public repositories already. I've
>>>> looked
>>>> > > into
>>>> > > >> trying to redirect references to local artifacts; however,
>>>> that's also
>>>> > > >> tricky since all the building occurs in Docker containers.
>>>> > > >>
>>>> > > >> Gordon, is there a trick to running the sample code in
>>>> > > >> flink-statefun-playground against yet-unreleased code that I'm
>>>> > missing?
>>>> > > >>
>>>> > > >> Thanks.
>>>> > > >>
>>>> > > >> On Sat, Jun 24, 2023 at 12:40 PM Galen Warren <
>>>> > galen@cvillewarrens.com>
>>>> > > >> wrote:
>>>> > > >>
>>>> > > >>> Great -- thanks!
>>>> > > >>>
>>>> > > >>> I'm going to be out of town for about a week but I'll take a
>>>> look at
>>>> > > >>> this when I'm back.
>>>> > > >>>
>>>> > > >>> On Tue, Jun 20, 2023 at 8:46 AM Martijn Visser <
>>>> mvisser@confluent.io
>>>> > >
>>>> > > >>> wrote:
>>>> > > >>>
>>>> > > >>>> Hi Galen,
>>>> > > >>>>
>>>> > > >>>> Yes, I'll be more than happy to help with Statefun releases.
>>>> > > >>>>
>>>> > > >>>> Best regards,
>>>> > > >>>>
>>>> > > >>>> Martijn
>>>> > > >>>>
>>>> > > >>>> On Tue, Jun 20, 2023 at 2:21 PM Galen Warren <
>>>> > galen@cvillewarrens.com
>>>> > > >
>>>> > > >>>> wrote:
>>>> > > >>>>
>>>> > > >>>>> Thanks.
>>>> > > >>>>>
>>>> > > >>>>> Martijn, to answer your question, I'd need to do a small
>>>> amount of
>>>> > > >>>>> work to get a PR ready, but not much. Happy to do it if we're
>>>> > > deciding to
>>>> > > >>>>> restart Statefun releases -- are we?
>>>> > > >>>>>
>>>> > > >>>>> -- Galen
>>>> > > >>>>>
>>>> > > >>>>> On Sat, Jun 17, 2023 at 9:47 AM Tzu-Li (Gordon) Tai <
>>>> > > >>>>> tzulitai@apache.org> wrote:
>>>> > > >>>>>
>>>> > > >>>>>> > Perhaps he could weigh in on whether the combination of
>>>> > automated
>>>> > > >>>>>> tests plus those smoke tests should be sufficient for
>>>> testing with
>>>> > > new
>>>> > > >>>>>> Flink versions
>>>> > > >>>>>>
>>>> > > >>>>>> What we usually did at the bare minimum for new StateFun
>>>> releases
>>>> > > was
>>>> > > >>>>>> the following:
>>>> > > >>>>>>
>>>> > > >>>>>>    1. Build tests (including the smoke tests in the e2e
>>>> module,
>>>> > > >>>>>>    which covers important tests like exactly-once
>>>> verification)
>>>> > > >>>>>>    2. Updating the flink-statefun-playground repo and
>>>> manually
>>>> > > >>>>>>    running all language examples there.
>>>> > > >>>>>>
>>>> > > >>>>>> If upgrading Flink versions was the only change in the
>>>> release,
>>>> > I'd
>>>> > > >>>>>> probably say that this is sufficient.
>>>> > > >>>>>>
>>>> > > >>>>>> Best,
>>>> > > >>>>>> Gordon
>>>> > > >>>>>>
>>>> > > >>>>>> On Thu, Jun 15, 2023 at 5:25 AM Martijn Visser <
>>>> > > >>>>>> martijnvisser@apache.org> wrote:
>>>> > > >>>>>>
>>>> > > >>>>>>> Let me know if you have a PR for a Flink update :)
>>>> > > >>>>>>>
>>>> > > >>>>>>> On Thu, Jun 8, 2023 at 5:52 PM Galen Warren via user <
>>>> > > >>>>>>> user@flink.apache.org> wrote:
>>>> > > >>>>>>>
>>>> > > >>>>>>>> Thanks Martijn.
>>>> > > >>>>>>>>
>>>> > > >>>>>>>> Personally, I'm already using a local fork of Statefun
>>>> that is
>>>> > > >>>>>>>> compatible with Flink 1.16.x, so I wouldn't have any need
>>>> for a
>>>> > > released
>>>> > > >>>>>>>> version compatible with 1.15.x. I'd be happy to do the PRs
>>>> to
>>>> > > modify
>>>> > > >>>>>>>> Statefun to work with new versions of Flink as they come
>>>> along.
>>>> > > >>>>>>>>
>>>> > > >>>>>>>> As for testing, Statefun does have unit tests and Gordon
>>>> also
>>>> > sent
>>>> > > >>>>>>>> me instructions a while back for how to do some additional
>>>> smoke
>>>> > > tests
>>>> > > >>>>>>>> which are pretty straightforward. Perhaps he could weigh
>>>> in on
>>>> > > whether the
>>>> > > >>>>>>>> combination of automated tests plus those smoke tests
>>>> should be
>>>> > > sufficient
>>>> > > >>>>>>>> for testing with new Flink versions (I believe the answer
>>>> is
>>>> > yes).
>>>> > > >>>>>>>>
>>>> > > >>>>>>>> -- Galen
>>>> > > >>>>>>>>
>>>> > > >>>>>>>>
>>>> > > >>>>>>>>
>>>> > > >>>>>>>> On Thu, Jun 8, 2023 at 8:01 AM Martijn Visser <
>>>> > > >>>>>>>> martijnvisser@apache.org> wrote:
>>>> > > >>>>>>>>
>>>> > > >>>>>>>>> Hi all,
>>>> > > >>>>>>>>>
>>>> > > >>>>>>>>> Apologies for the late reply.
>>>> > > >>>>>>>>>
>>>> > > >>>>>>>>> I'm willing to help out with merging requests in Statefun
>>>> to
>>>> > keep
>>>> > > >>>>>>>>> them
>>>> > > >>>>>>>>> compatible with new Flink releases and create new
>>>> releases. I
>>>> > do
>>>> > > >>>>>>>>> think that
>>>> > > >>>>>>>>> validation of the functionality of these releases depends
>>>> a lot
>>>> > > on
>>>> > > >>>>>>>>> those
>>>> > > >>>>>>>>> who do these compatibility updates, with PMC members
>>>> helping
>>>> > out
>>>> > > >>>>>>>>> with the
>>>> > > >>>>>>>>> formal process.
>>>> > > >>>>>>>>>
>>>> > > >>>>>>>>> > Why can't the Apache Software Foundation allow community
>>>> > > members
>>>> > > >>>>>>>>> to bring
>>>> > > >>>>>>>>> it up to date?
>>>> > > >>>>>>>>>
>>>> > > >>>>>>>>> There's nothing preventing anyone from reviewing any of
>>>> the
>>>> > > >>>>>>>>> current PRs or
>>>> > > >>>>>>>>> opening new ones. However, none of them are approved [1],
>>>> so
>>>> > > >>>>>>>>> there's also
>>>> > > >>>>>>>>> nothing to merge.
>>>> > > >>>>>>>>>
>>>> > > >>>>>>>>> > I believe that there are people and companies on this
>>>> mailing
>>>> > > >>>>>>>>> list
>>>> > > >>>>>>>>> interested in supporting Apache Flink Stateful Functions.
>>>> > > >>>>>>>>>
>>>> > > >>>>>>>>> If so, then now is the time to show.
>>>> > > >>>>>>>>>
>>>> > > >>>>>>>>> Would there be a preference to create a release with
>>>> Galen's
>>>> > > merged
>>>> > > >>>>>>>>> compatibility update to Flink 1.15.2, or do we want to
>>>> skip
>>>> > that
>>>> > > >>>>>>>>> and go
>>>> > > >>>>>>>>> straight to a newer version?
>>>> > > >>>>>>>>>
>>>> > > >>>>>>>>> Best regards,
>>>> > > >>>>>>>>>
>>>> > > >>>>>>>>> Martijn
>>>> > > >>>>>>>>>
>>>> > > >>>>>>>>> [1]
>>>> > > >>>>>>>>>
>>>> > > >>>>>>>>>
>>>> > >
>>>> >
>>>> https://github.com/apache/flink-statefun/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved
>>>> > > >>>>>>>>>
>>>> > > >>>>>>>>> On Tue, Jun 6, 2023 at 3:55 PM Marco Villalobos <
>>>> > > >>>>>>>>> mvillalobos@kineteque.com>
>>>> > > >>>>>>>>> wrote:
>>>> > > >>>>>>>>>
>>>> > > >>>>>>>>> > Why can't the Apache Software Foundation allow community
>>>> > > members
>>>> > > >>>>>>>>> to bring
>>>> > > >>>>>>>>> > it up to date?
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > What's the process for that?
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > I believe that there are people and companies on this
>>>> mailing
>>>> > > >>>>>>>>> list
>>>> > > >>>>>>>>> > interested in supporting Apache Flink Stateful
>>>> Functions.
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > You already had two people on this thread express
>>>> interest.
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > At the very least, we could keep the library versions
>>>> up to
>>>> > > date.
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > There are only a small list of new features that might
>>>> be
>>>> > > >>>>>>>>> worthwhile:
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > 1. event time processing
>>>> > > >>>>>>>>> > 2. state rest api
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <
>>>> > > chesnay@apache.org>
>>>> > > >>>>>>>>> wrote:
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > If you were to fork it *and want to redistribute it*
>>>> then the
>>>> > > >>>>>>>>> short
>>>> > > >>>>>>>>> > version is that
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> >    1. you have to adhere to the Apache licensing
>>>> requirements
>>>> > > >>>>>>>>> >    2. you have to make it clear that your fork does not
>>>> > belong
>>>> > > >>>>>>>>> to the
>>>> > > >>>>>>>>> >    Apache Flink project. (Trademarks and all that)
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > Neither should be significant hurdles (there should
>>>> also be
>>>> > > >>>>>>>>> plenty of
>>>> > > >>>>>>>>> > online resources regarding 1), and if you do this then
>>>> you
>>>> > can
>>>> > > >>>>>>>>> freely share
>>>> > > >>>>>>>>> > your fork with others.
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > I've also pinged Martijn to take a look at this thread.
>>>> > > >>>>>>>>> > To my knowledge the project hasn't decided anything yet.
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > On 27/05/2023 04:05, Galen Warren wrote:
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > Ok, I get it. No interest.
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > If this project is being abandoned, I guess I'll work
>>>> with my
>>>> > > >>>>>>>>> own fork. Is
>>>> > > >>>>>>>>> > there anything I should consider here? Can I share it
>>>> with
>>>> > > other
>>>> > > >>>>>>>>> people who
>>>> > > >>>>>>>>> > use this project?
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > On Tue, May 16, 2023 at 10:50 AM Galen Warren <
>>>> > > >>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
>>>> > > >>>>>>>>> > wrote:
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > Hi Martijn, since you opened this discussion thread, I'm
>>>> > > curious
>>>> > > >>>>>>>>> what your
>>>> > > >>>>>>>>> > thoughts are in light of the responses? Thanks.
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <
>>>> > > >>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
>>>> > > >>>>>>>>> > wrote:
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > I use Apache Flink for stream processing, and StateFun
>>>> as a
>>>> > > >>>>>>>>> hand-off
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > point for the rest of the application.
>>>> > > >>>>>>>>> > It serves well as a bridge between a Flink Streaming
>>>> job and
>>>> > > >>>>>>>>> > micro-services.
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > This is essentially how I use it as well, and I would
>>>> also be
>>>> > > >>>>>>>>> sad to see
>>>> > > >>>>>>>>> > it sunsetted. It works well; I don't know that there is
>>>> a lot
>>>> > > of
>>>> > > >>>>>>>>> new
>>>> > > >>>>>>>>> > development required, but if there are no new Statefun
>>>> > > releases,
>>>> > > >>>>>>>>> then
>>>> > > >>>>>>>>> > Statefun can only be used with older Flink versions.
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
>>>> > > >>>>>>>>> mvillalobos@kineteque.com> wrote:
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > I am currently using Stateful Functions in my
>>>> application.
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > I use Apache Flink for stream processing, and StateFun
>>>> as a
>>>> > > >>>>>>>>> hand-off
>>>> > > >>>>>>>>> > point for the rest of the application.
>>>> > > >>>>>>>>> > It serves well as a bridge between a Flink Streaming
>>>> job and
>>>> > > >>>>>>>>> > micro-services.
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > I would be disappointed if StateFun was sunsetted.  Its
>>>> a
>>>> > good
>>>> > > >>>>>>>>> idea.
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > If there is anything I can do to help, as a contributor
>>>> > > perhaps,
>>>> > > >>>>>>>>> please
>>>> > > >>>>>>>>> > let me know.
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <
>>>> > > >>>>>>>>> martijnvisser@apache.org> <ma...@apache.org>
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > wrote:
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > Hi everyone,
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > I want to open a discussion on the status of the
>>>> Statefun
>>>> > > >>>>>>>>> Project [1]
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > in Apache Flink. As you might have noticed, there
>>>> hasn't been
>>>> > > >>>>>>>>> much
>>>> > > >>>>>>>>> > development over the past months in the Statefun
>>>> repository
>>>> > > [2].
>>>> > > >>>>>>>>> There is
>>>> > > >>>>>>>>> > currently a lack of active contributors and committers
>>>> who
>>>> > are
>>>> > > >>>>>>>>> able to help
>>>> > > >>>>>>>>> > with the maintenance of the project.
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > In order to improve the situation, we need to solve the
>>>> lack
>>>> > of
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > committers and the lack of contributors.
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > On the lack of committers:
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > 1. Ideally, there are some of the current Flink
>>>> committers
>>>> > who
>>>> > > >>>>>>>>> have
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > the bandwidth and can help with reviewing PRs and
>>>> merging
>>>> > them.
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > 2. If that's not an option, it could be a consideration
>>>> that
>>>> > > >>>>>>>>> current
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > committers only approve and review PRs, that are
>>>> approved by
>>>> > > >>>>>>>>> those who are
>>>> > > >>>>>>>>> > willing to contribute to Statefun and if the CI passes
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > On the lack of contributors:
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > 3. Next to having this discussion on the Dev and User
>>>> mailing
>>>> > > >>>>>>>>> list, we
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > can also create a blog with a call for new contributors
>>>> on
>>>> > the
>>>> > > >>>>>>>>> Flink
>>>> > > >>>>>>>>> > project website, send out some tweets on the Flink /
>>>> Statefun
>>>> > > >>>>>>>>> twitter
>>>> > > >>>>>>>>> > accounts, post messages on Slack etc. In that message,
>>>> we
>>>> > would
>>>> > > >>>>>>>>> inform how
>>>> > > >>>>>>>>> > those that are interested in contributing can start and
>>>> where
>>>> > > >>>>>>>>> they could
>>>> > > >>>>>>>>> > reach out for more information.
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > There's also option 4. where a group of interested
>>>> people
>>>> > would
>>>> > > >>>>>>>>> split
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > Statefun from the Flink project and make it a separate
>>>> top
>>>> > > level
>>>> > > >>>>>>>>> project
>>>> > > >>>>>>>>> > under the Apache Flink umbrella (similar as recently has
>>>> > > >>>>>>>>> happened with
>>>> > > >>>>>>>>> > Flink Table Store, which has become Apache Paimon).
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > If we see no improvements in the coming period, we
>>>> should
>>>> > > >>>>>>>>> consider
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > sunsetting Statefun and communicate that clearly to the
>>>> > users.
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > I'm looking forward to your thoughts.
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > Best regards,
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > Martijn
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > [1]
>>>> > > >>>>>>>>>
>>>> https://nightlies.apache.org/flink/flink-statefun-docs-master/
>>>> > <
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> >
>>>> > https://nightlies.apache.org/flink/flink-statefun-docs-master/
>>>> > > >
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > [2] https://github.com/apache/flink-statefun <
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > https://github.com/apache/flink-statefun>
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>>
>>>> > > >>>>>>>>
>>>> > >
>>>> >
>>>>
>>>

Re: [DISCUSS] Status of Statefun Project

[Filip Karnicki <fi...@gmail.com>]

Hi Gordon

Any chance we could get this reviewed and released to the central repo?
We're currently forced to use a Flink version that has a nasty bug causing
an operational nightmare

Many thanks
Fil

On Sat, 19 Aug 2023 at 01:38, Galen Warren via user <us...@flink.apache.org>
wrote:

> Gotcha, makes sense as to the original division.
>
> >> Can this be solved by simply passing in the path to the artifacts
>
> This definitely works if we're going to be copying the artifacts on the
> host side -- into the build context -- and then from the context into the
> image. It only gets tricky to have a potentially varying path to the
> artifacts if we're trying to *directly *include the artifacts in the
> Docker context -- then we have a situation where the Docker context must
> contain both the artifacts and playground files, with (potentially)
> different root locations.
>
> Maybe the simplest thing to do here is just to leave the playground as-is
> and then copy the artifacts into the Docker context manually, prior to
> building the playground images. I'm fine with that. It will mean that each
> Statefun release will require two PRs and two sets of build/publish steps
> instead of one, but if everyone else is fine with that I am, too. Unless
> anyone objects, I'll go ahead and queue up a PR for the playground that
> makes these changes.
>
> Also, I should mention -- in case it's not clear -- that I have already
> built and run the playground examples with the code from the PR and
> everything worked. So that PR is ready to move forward with review, etc.,
> at this point.
>
> Thanks.
>
>
>
>
>
>
>
> On Fri, Aug 18, 2023 at 4:16 PM Tzu-Li (Gordon) Tai <tz...@apache.org>
> wrote:
>
>> Hi Galen,
>>
>> The original intent of having a separate repo for the playground repo,
>> was that StateFun users can just go to that and start running simple
>> examples without any other distractions from the core code. I personally
>> don't have a strong preference here and can understand how it would make
>> the workflow more streamlined, but just FYI on the reasoning why are
>> separate in the first place.
>>
>> re: paths for locating StateFun artifacts.
>> Can this be solved by simply passing in the path to the artifacts? As
>> well as the image tag for the locally build base StateFun image. They could
>> probably be environment variables.
>>
>> Cheers,
>> Gordon
>>
>> On Fri, Aug 18, 2023 at 12:13 PM Galen Warren via user <
>> user@flink.apache.org> wrote:
>>
>>> Yes, exactly! And in addition to the base Statefun jars and the jar for
>>> the Java SDK, it does an equivalent copy/register operation for each of the
>>> other SDK libraries (Go, Python, Javascript) so that those libraries are
>>> also available when building the playground examples.
>>>
>>> One more question: In order to copy the various build artifacts into the
>>> Docker containers, those artifacts need to be part of the Docker context.
>>> With the playground being a separate project, that's slightly tricky to do,
>>> as there is no guarantee (other than convention) about the relative paths
>>> of *flink-statefun* and* flink-statefun-playground *in someone's local
>>> filesystem. The way I've set this up locally is to copy the playground into
>>> the* flink-statefun* project -- i.e. to *flink-statefun*/playground --
>>> such that I can set the Docker context to the root folder of
>>> *flink-statefun* and then have access to any local code and/or build
>>> artifacts.
>>>
>>> I'm wondering if there might be any appetite for making that move
>>> permanent, i.e. moving the playground to *flink-statefun*/playground
>>> and deprecating the standalone playground project. In addition to making
>>> the problem of building with unreleased artifacts a bit simpler to solve,
>>> it would also simplify the process of releasing a new Statefun version,
>>> since the entire process could be handled with a single PR and associated
>>> build/deploy tasks. In other words, a single PR could both update and
>>> deploy the Statefun package and the playground code and images.
>>>
>>> As it stands, at least two PRs would be required for each Statefun
>>> version update -- one for *flink-statefun* and one for
>>> *flink-statefun-playground*.
>>>
>>> Anyway, just an idea. Maybe there's an important reason for these
>>> projects to remain separate. If we do want to keep the playground project
>>> where it is, I could solve the copying problem by requiring the two
>>> projects to be siblings in the file system and by pre-copying the local
>>> build artifacts into a location accessible by the existing Docker contexts.
>>> This would still leave us with the need to have two PRs and releases
>>> instead of one, though.
>>>
>>> Thanks for your help!
>>>
>>>
>>> On Fri, Aug 18, 2023 at 2:45 PM Tzu-Li (Gordon) Tai <tz...@apache.org>
>>> wrote:
>>>
>>>> Hi Galen,
>>>>
>>>> > locally built code is copied into the build containers
>>>> so that it can be accessed during the build.
>>>>
>>>> That's exactly what we had been doing for release testing, yes. Sorry I
>>>> missed that detail in my previous response.
>>>>
>>>> And yes, that sounds like a reasonable approach. If I understand you
>>>> correctly, the workflow would become this:
>>>>
>>>>    1. Build the StateFun repo locally to install the snapshot artifact
>>>>    jars + have a local base StateFun image.
>>>>    2. Run the playground in "local" mode, so that it uses the local base
>>>>    StateFun image + builds the playground code using copied artifact
>>>> jars
>>>>    (instead of pulling from Maven).
>>>>
>>>> That looks good to me!
>>>>
>>>> Thanks,
>>>> Gordon
>>>>
>>>> On Fri, Aug 18, 2023 at 11:33 AM Galen Warren
>>>> <ga...@cvillewarrens.com.invalid> wrote:
>>>>
>>>> > Thanks.
>>>> >
>>>> > If you were to build a local image, as you suggest, how do you access
>>>> that
>>>> > image when building the playground images? All the compilation of
>>>> > playground code happens inside containers, so local images on the host
>>>> > aren't available in those containers. Unless I'm missing something?
>>>> >
>>>> > I've slightly reworked things such that the playground images can be
>>>> run in
>>>> > one of two modes -- the default mode, which works like before, and a
>>>> > "local" mode where locally built code is copied into the build
>>>> containers
>>>> > so that it can be accessed during the build. It works fine, you just
>>>> have
>>>> > to define a couple of environment variables when running
>>>> docker-compose to
>>>> > specify default vs. local mode and what versions of Flink and Statefun
>>>> > should be referenced, and then you can build a run the local examples
>>>> > without any additional steps. Does that sound like a reasonable
>>>> approach?
>>>> >
>>>> >
>>>> > On Fri, Aug 18, 2023 at 2:17 PM Tzu-Li (Gordon) Tai <
>>>> tzulitai@apache.org>
>>>> > wrote:
>>>> >
>>>> > > Hi Galen,
>>>> > >
>>>> > > > Gordon, is there a trick to running the sample code in
>>>> > > flink-statefun-playground against yet-unreleased code that I'm
>>>> missing?
>>>> > >
>>>> > > You'd have to locally build an image from the release branch, with a
>>>> > > temporary image version tag. Then, in the flink-statefun-playground,
>>>> > change
>>>> > > the image versions in the docker-compose files to use that locally
>>>> built
>>>> > > image. IIRC, that's what we have been doing in the past.
>>>> Admittedly, it's
>>>> > > pretty manual - I don't think the CI manages this workflow.
>>>> > >
>>>> > > Thanks,
>>>> > > Gordon
>>>> > >
>>>> > > On Mon, Aug 14, 2023 at 10:42 AM Galen Warren <
>>>> galen@cvillewarrens.com>
>>>> > > wrote:
>>>> > >
>>>> > > > I created a pull request for this: [FLINK-31619] Upgrade Stateful
>>>> > > > Functions to Flink 1.16.1 by galenwarren · Pull Request #331 ·
>>>> > > > apache/flink-statefun (github.com)
>>>> > > > <https://github.com/apache/flink-statefun/pull/331>.
>>>> > > >
>>>> > > > JIRA is here: [FLINK-31619] Upgrade Stateful Functions to Flink
>>>> 1.16.1
>>>> > -
>>>> > > > ASF JIRA (apache.org)
>>>> > > > <https://issues.apache.org/jira/browse/FLINK-31619?filter=-1>.
>>>> > > >
>>>> > > > Statefun references 1.16.2, despite the title -- that version has
>>>> come
>>>> > > out
>>>> > > > since the issue was created.
>>>> > > >
>>>> > > > I figured out how to run all the playground tests locally, but it
>>>> took
>>>> > a
>>>> > > > bit of reworking of the playground setup with respect to Docker;
>>>> > > > specifically, the Docker contexts used to build the example
>>>> functions
>>>> > > > needed to be broadened (i.e. moved up the tree) so that, if
>>>> needed,
>>>> > local
>>>> > > > artifacts/code can be accessed from within the containers at build
>>>> > time.
>>>> > > > Then I made the Docker compose.yml configurable through
>>>> environment
>>>> > > > variables to allow for them to run in either the original manner
>>>> --
>>>> > i.e.
>>>> > > > pulling artifacts from public repos -- or in a "local" mode, where
>>>> > > > artifacts are pulled from local builds.
>>>> > > >
>>>> > > > This process is a cleaner if the playground is a subfolder of the
>>>> > > > flink-statefun project rather than be its own repository
>>>> > > > (flink-statefun-playground), because then all the relative paths
>>>> > between
>>>> > > > the playground files and the build artifacts are fixed. So, I'd
>>>> like to
>>>> > > > propose to move the playground files, modified as described
>>>> above, to
>>>> > > > flink-statefun/playground and retire flink-statefun-playground. I
>>>> can
>>>> > > > submit separate PR s those changes if everyone is on board.
>>>> > > >
>>>> > > > Also, should I plan to do the same upgrade to handle Flink
>>>> 1.17.x? It
>>>> > > > should be easy to do, especially while the 1.16.x upgrade is
>>>> fresh on
>>>> > my
>>>> > > > mind.
>>>> > > >
>>>> > > > Thanks.
>>>> > > >
>>>> > > >
>>>> > > > On Fri, Aug 11, 2023 at 6:40 PM Galen Warren <
>>>> galen@cvillewarrens.com>
>>>> > > > wrote:
>>>> > > >
>>>> > > >> I'm done with the code to make Statefun compatible with Flink
>>>> 1.16,
>>>> > and
>>>> > > >> all the tests (including e2e succeed). The required changes were
>>>> > pretty
>>>> > > >> minimal.
>>>> > > >>
>>>> > > >> I'm running into a bit of a chicken/egg problem executing the
>>>> tests in
>>>> > > >> flink-statefun-playground
>>>> > > >> <https://github.com/apache/flink-statefun-playground>, though.
>>>> That
>>>> > > >> project seems to assume that all the various Statefun artifacts
>>>> are
>>>> > > built
>>>> > > >> and deployed to the various public repositories already. I've
>>>> looked
>>>> > > into
>>>> > > >> trying to redirect references to local artifacts; however,
>>>> that's also
>>>> > > >> tricky since all the building occurs in Docker containers.
>>>> > > >>
>>>> > > >> Gordon, is there a trick to running the sample code in
>>>> > > >> flink-statefun-playground against yet-unreleased code that I'm
>>>> > missing?
>>>> > > >>
>>>> > > >> Thanks.
>>>> > > >>
>>>> > > >> On Sat, Jun 24, 2023 at 12:40 PM Galen Warren <
>>>> > galen@cvillewarrens.com>
>>>> > > >> wrote:
>>>> > > >>
>>>> > > >>> Great -- thanks!
>>>> > > >>>
>>>> > > >>> I'm going to be out of town for about a week but I'll take a
>>>> look at
>>>> > > >>> this when I'm back.
>>>> > > >>>
>>>> > > >>> On Tue, Jun 20, 2023 at 8:46 AM Martijn Visser <
>>>> mvisser@confluent.io
>>>> > >
>>>> > > >>> wrote:
>>>> > > >>>
>>>> > > >>>> Hi Galen,
>>>> > > >>>>
>>>> > > >>>> Yes, I'll be more than happy to help with Statefun releases.
>>>> > > >>>>
>>>> > > >>>> Best regards,
>>>> > > >>>>
>>>> > > >>>> Martijn
>>>> > > >>>>
>>>> > > >>>> On Tue, Jun 20, 2023 at 2:21 PM Galen Warren <
>>>> > galen@cvillewarrens.com
>>>> > > >
>>>> > > >>>> wrote:
>>>> > > >>>>
>>>> > > >>>>> Thanks.
>>>> > > >>>>>
>>>> > > >>>>> Martijn, to answer your question, I'd need to do a small
>>>> amount of
>>>> > > >>>>> work to get a PR ready, but not much. Happy to do it if we're
>>>> > > deciding to
>>>> > > >>>>> restart Statefun releases -- are we?
>>>> > > >>>>>
>>>> > > >>>>> -- Galen
>>>> > > >>>>>
>>>> > > >>>>> On Sat, Jun 17, 2023 at 9:47 AM Tzu-Li (Gordon) Tai <
>>>> > > >>>>> tzulitai@apache.org> wrote:
>>>> > > >>>>>
>>>> > > >>>>>> > Perhaps he could weigh in on whether the combination of
>>>> > automated
>>>> > > >>>>>> tests plus those smoke tests should be sufficient for
>>>> testing with
>>>> > > new
>>>> > > >>>>>> Flink versions
>>>> > > >>>>>>
>>>> > > >>>>>> What we usually did at the bare minimum for new StateFun
>>>> releases
>>>> > > was
>>>> > > >>>>>> the following:
>>>> > > >>>>>>
>>>> > > >>>>>>    1. Build tests (including the smoke tests in the e2e
>>>> module,
>>>> > > >>>>>>    which covers important tests like exactly-once
>>>> verification)
>>>> > > >>>>>>    2. Updating the flink-statefun-playground repo and
>>>> manually
>>>> > > >>>>>>    running all language examples there.
>>>> > > >>>>>>
>>>> > > >>>>>> If upgrading Flink versions was the only change in the
>>>> release,
>>>> > I'd
>>>> > > >>>>>> probably say that this is sufficient.
>>>> > > >>>>>>
>>>> > > >>>>>> Best,
>>>> > > >>>>>> Gordon
>>>> > > >>>>>>
>>>> > > >>>>>> On Thu, Jun 15, 2023 at 5:25 AM Martijn Visser <
>>>> > > >>>>>> martijnvisser@apache.org> wrote:
>>>> > > >>>>>>
>>>> > > >>>>>>> Let me know if you have a PR for a Flink update :)
>>>> > > >>>>>>>
>>>> > > >>>>>>> On Thu, Jun 8, 2023 at 5:52 PM Galen Warren via user <
>>>> > > >>>>>>> user@flink.apache.org> wrote:
>>>> > > >>>>>>>
>>>> > > >>>>>>>> Thanks Martijn.
>>>> > > >>>>>>>>
>>>> > > >>>>>>>> Personally, I'm already using a local fork of Statefun
>>>> that is
>>>> > > >>>>>>>> compatible with Flink 1.16.x, so I wouldn't have any need
>>>> for a
>>>> > > released
>>>> > > >>>>>>>> version compatible with 1.15.x. I'd be happy to do the PRs
>>>> to
>>>> > > modify
>>>> > > >>>>>>>> Statefun to work with new versions of Flink as they come
>>>> along.
>>>> > > >>>>>>>>
>>>> > > >>>>>>>> As for testing, Statefun does have unit tests and Gordon
>>>> also
>>>> > sent
>>>> > > >>>>>>>> me instructions a while back for how to do some additional
>>>> smoke
>>>> > > tests
>>>> > > >>>>>>>> which are pretty straightforward. Perhaps he could weigh
>>>> in on
>>>> > > whether the
>>>> > > >>>>>>>> combination of automated tests plus those smoke tests
>>>> should be
>>>> > > sufficient
>>>> > > >>>>>>>> for testing with new Flink versions (I believe the answer
>>>> is
>>>> > yes).
>>>> > > >>>>>>>>
>>>> > > >>>>>>>> -- Galen
>>>> > > >>>>>>>>
>>>> > > >>>>>>>>
>>>> > > >>>>>>>>
>>>> > > >>>>>>>> On Thu, Jun 8, 2023 at 8:01 AM Martijn Visser <
>>>> > > >>>>>>>> martijnvisser@apache.org> wrote:
>>>> > > >>>>>>>>
>>>> > > >>>>>>>>> Hi all,
>>>> > > >>>>>>>>>
>>>> > > >>>>>>>>> Apologies for the late reply.
>>>> > > >>>>>>>>>
>>>> > > >>>>>>>>> I'm willing to help out with merging requests in Statefun
>>>> to
>>>> > keep
>>>> > > >>>>>>>>> them
>>>> > > >>>>>>>>> compatible with new Flink releases and create new
>>>> releases. I
>>>> > do
>>>> > > >>>>>>>>> think that
>>>> > > >>>>>>>>> validation of the functionality of these releases depends
>>>> a lot
>>>> > > on
>>>> > > >>>>>>>>> those
>>>> > > >>>>>>>>> who do these compatibility updates, with PMC members
>>>> helping
>>>> > out
>>>> > > >>>>>>>>> with the
>>>> > > >>>>>>>>> formal process.
>>>> > > >>>>>>>>>
>>>> > > >>>>>>>>> > Why can't the Apache Software Foundation allow community
>>>> > > members
>>>> > > >>>>>>>>> to bring
>>>> > > >>>>>>>>> it up to date?
>>>> > > >>>>>>>>>
>>>> > > >>>>>>>>> There's nothing preventing anyone from reviewing any of
>>>> the
>>>> > > >>>>>>>>> current PRs or
>>>> > > >>>>>>>>> opening new ones. However, none of them are approved [1],
>>>> so
>>>> > > >>>>>>>>> there's also
>>>> > > >>>>>>>>> nothing to merge.
>>>> > > >>>>>>>>>
>>>> > > >>>>>>>>> > I believe that there are people and companies on this
>>>> mailing
>>>> > > >>>>>>>>> list
>>>> > > >>>>>>>>> interested in supporting Apache Flink Stateful Functions.
>>>> > > >>>>>>>>>
>>>> > > >>>>>>>>> If so, then now is the time to show.
>>>> > > >>>>>>>>>
>>>> > > >>>>>>>>> Would there be a preference to create a release with
>>>> Galen's
>>>> > > merged
>>>> > > >>>>>>>>> compatibility update to Flink 1.15.2, or do we want to
>>>> skip
>>>> > that
>>>> > > >>>>>>>>> and go
>>>> > > >>>>>>>>> straight to a newer version?
>>>> > > >>>>>>>>>
>>>> > > >>>>>>>>> Best regards,
>>>> > > >>>>>>>>>
>>>> > > >>>>>>>>> Martijn
>>>> > > >>>>>>>>>
>>>> > > >>>>>>>>> [1]
>>>> > > >>>>>>>>>
>>>> > > >>>>>>>>>
>>>> > >
>>>> >
>>>> https://github.com/apache/flink-statefun/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved
>>>> > > >>>>>>>>>
>>>> > > >>>>>>>>> On Tue, Jun 6, 2023 at 3:55 PM Marco Villalobos <
>>>> > > >>>>>>>>> mvillalobos@kineteque.com>
>>>> > > >>>>>>>>> wrote:
>>>> > > >>>>>>>>>
>>>> > > >>>>>>>>> > Why can't the Apache Software Foundation allow community
>>>> > > members
>>>> > > >>>>>>>>> to bring
>>>> > > >>>>>>>>> > it up to date?
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > What's the process for that?
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > I believe that there are people and companies on this
>>>> mailing
>>>> > > >>>>>>>>> list
>>>> > > >>>>>>>>> > interested in supporting Apache Flink Stateful
>>>> Functions.
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > You already had two people on this thread express
>>>> interest.
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > At the very least, we could keep the library versions
>>>> up to
>>>> > > date.
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > There are only a small list of new features that might
>>>> be
>>>> > > >>>>>>>>> worthwhile:
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > 1. event time processing
>>>> > > >>>>>>>>> > 2. state rest api
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <
>>>> > > chesnay@apache.org>
>>>> > > >>>>>>>>> wrote:
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > If you were to fork it *and want to redistribute it*
>>>> then the
>>>> > > >>>>>>>>> short
>>>> > > >>>>>>>>> > version is that
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> >    1. you have to adhere to the Apache licensing
>>>> requirements
>>>> > > >>>>>>>>> >    2. you have to make it clear that your fork does not
>>>> > belong
>>>> > > >>>>>>>>> to the
>>>> > > >>>>>>>>> >    Apache Flink project. (Trademarks and all that)
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > Neither should be significant hurdles (there should
>>>> also be
>>>> > > >>>>>>>>> plenty of
>>>> > > >>>>>>>>> > online resources regarding 1), and if you do this then
>>>> you
>>>> > can
>>>> > > >>>>>>>>> freely share
>>>> > > >>>>>>>>> > your fork with others.
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > I've also pinged Martijn to take a look at this thread.
>>>> > > >>>>>>>>> > To my knowledge the project hasn't decided anything yet.
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > On 27/05/2023 04:05, Galen Warren wrote:
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > Ok, I get it. No interest.
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > If this project is being abandoned, I guess I'll work
>>>> with my
>>>> > > >>>>>>>>> own fork. Is
>>>> > > >>>>>>>>> > there anything I should consider here? Can I share it
>>>> with
>>>> > > other
>>>> > > >>>>>>>>> people who
>>>> > > >>>>>>>>> > use this project?
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > On Tue, May 16, 2023 at 10:50 AM Galen Warren <
>>>> > > >>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
>>>> > > >>>>>>>>> > wrote:
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > Hi Martijn, since you opened this discussion thread, I'm
>>>> > > curious
>>>> > > >>>>>>>>> what your
>>>> > > >>>>>>>>> > thoughts are in light of the responses? Thanks.
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <
>>>> > > >>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
>>>> > > >>>>>>>>> > wrote:
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > I use Apache Flink for stream processing, and StateFun
>>>> as a
>>>> > > >>>>>>>>> hand-off
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > point for the rest of the application.
>>>> > > >>>>>>>>> > It serves well as a bridge between a Flink Streaming
>>>> job and
>>>> > > >>>>>>>>> > micro-services.
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > This is essentially how I use it as well, and I would
>>>> also be
>>>> > > >>>>>>>>> sad to see
>>>> > > >>>>>>>>> > it sunsetted. It works well; I don't know that there is
>>>> a lot
>>>> > > of
>>>> > > >>>>>>>>> new
>>>> > > >>>>>>>>> > development required, but if there are no new Statefun
>>>> > > releases,
>>>> > > >>>>>>>>> then
>>>> > > >>>>>>>>> > Statefun can only be used with older Flink versions.
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
>>>> > > >>>>>>>>> mvillalobos@kineteque.com> wrote:
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > I am currently using Stateful Functions in my
>>>> application.
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > I use Apache Flink for stream processing, and StateFun
>>>> as a
>>>> > > >>>>>>>>> hand-off
>>>> > > >>>>>>>>> > point for the rest of the application.
>>>> > > >>>>>>>>> > It serves well as a bridge between a Flink Streaming
>>>> job and
>>>> > > >>>>>>>>> > micro-services.
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > I would be disappointed if StateFun was sunsetted.  Its
>>>> a
>>>> > good
>>>> > > >>>>>>>>> idea.
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > If there is anything I can do to help, as a contributor
>>>> > > perhaps,
>>>> > > >>>>>>>>> please
>>>> > > >>>>>>>>> > let me know.
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <
>>>> > > >>>>>>>>> martijnvisser@apache.org> <ma...@apache.org>
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > wrote:
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > Hi everyone,
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > I want to open a discussion on the status of the
>>>> Statefun
>>>> > > >>>>>>>>> Project [1]
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > in Apache Flink. As you might have noticed, there
>>>> hasn't been
>>>> > > >>>>>>>>> much
>>>> > > >>>>>>>>> > development over the past months in the Statefun
>>>> repository
>>>> > > [2].
>>>> > > >>>>>>>>> There is
>>>> > > >>>>>>>>> > currently a lack of active contributors and committers
>>>> who
>>>> > are
>>>> > > >>>>>>>>> able to help
>>>> > > >>>>>>>>> > with the maintenance of the project.
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > In order to improve the situation, we need to solve the
>>>> lack
>>>> > of
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > committers and the lack of contributors.
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > On the lack of committers:
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > 1. Ideally, there are some of the current Flink
>>>> committers
>>>> > who
>>>> > > >>>>>>>>> have
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > the bandwidth and can help with reviewing PRs and
>>>> merging
>>>> > them.
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > 2. If that's not an option, it could be a consideration
>>>> that
>>>> > > >>>>>>>>> current
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > committers only approve and review PRs, that are
>>>> approved by
>>>> > > >>>>>>>>> those who are
>>>> > > >>>>>>>>> > willing to contribute to Statefun and if the CI passes
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > On the lack of contributors:
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > 3. Next to having this discussion on the Dev and User
>>>> mailing
>>>> > > >>>>>>>>> list, we
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > can also create a blog with a call for new contributors
>>>> on
>>>> > the
>>>> > > >>>>>>>>> Flink
>>>> > > >>>>>>>>> > project website, send out some tweets on the Flink /
>>>> Statefun
>>>> > > >>>>>>>>> twitter
>>>> > > >>>>>>>>> > accounts, post messages on Slack etc. In that message,
>>>> we
>>>> > would
>>>> > > >>>>>>>>> inform how
>>>> > > >>>>>>>>> > those that are interested in contributing can start and
>>>> where
>>>> > > >>>>>>>>> they could
>>>> > > >>>>>>>>> > reach out for more information.
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > There's also option 4. where a group of interested
>>>> people
>>>> > would
>>>> > > >>>>>>>>> split
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > Statefun from the Flink project and make it a separate
>>>> top
>>>> > > level
>>>> > > >>>>>>>>> project
>>>> > > >>>>>>>>> > under the Apache Flink umbrella (similar as recently has
>>>> > > >>>>>>>>> happened with
>>>> > > >>>>>>>>> > Flink Table Store, which has become Apache Paimon).
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > If we see no improvements in the coming period, we
>>>> should
>>>> > > >>>>>>>>> consider
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > sunsetting Statefun and communicate that clearly to the
>>>> > users.
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > I'm looking forward to your thoughts.
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > Best regards,
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > Martijn
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > [1]
>>>> > > >>>>>>>>>
>>>> https://nightlies.apache.org/flink/flink-statefun-docs-master/
>>>> > <
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> >
>>>> > https://nightlies.apache.org/flink/flink-statefun-docs-master/
>>>> > > >
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > [2] https://github.com/apache/flink-statefun <
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> > https://github.com/apache/flink-statefun>
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>> >
>>>> > > >>>>>>>>>
>>>> > > >>>>>>>>
>>>> > >
>>>> >
>>>>
>>>

Re: [DISCUSS] Status of Statefun Project

[Galen Warren <ga...@cvillewarrens.com.INVALID>]

Gotcha, makes sense as to the original division.

>> Can this be solved by simply passing in the path to the artifacts

This definitely works if we're going to be copying the artifacts on the
host side -- into the build context -- and then from the context into the
image. It only gets tricky to have a potentially varying path to the
artifacts if we're trying to *directly *include the artifacts in the Docker
context -- then we have a situation where the Docker context must contain
both the artifacts and playground files, with (potentially) different root
locations.

Maybe the simplest thing to do here is just to leave the playground as-is
and then copy the artifacts into the Docker context manually, prior to
building the playground images. I'm fine with that. It will mean that each
Statefun release will require two PRs and two sets of build/publish steps
instead of one, but if everyone else is fine with that I am, too. Unless
anyone objects, I'll go ahead and queue up a PR for the playground that
makes these changes.

Also, I should mention -- in case it's not clear -- that I have already
built and run the playground examples with the code from the PR and
everything worked. So that PR is ready to move forward with review, etc.,
at this point.

Thanks.







On Fri, Aug 18, 2023 at 4:16 PM Tzu-Li (Gordon) Tai <tz...@apache.org>
wrote:

> Hi Galen,
>
> The original intent of having a separate repo for the playground repo, was
> that StateFun users can just go to that and start running simple examples
> without any other distractions from the core code. I personally don't have
> a strong preference here and can understand how it would make the workflow
> more streamlined, but just FYI on the reasoning why are separate in the
> first place.
>
> re: paths for locating StateFun artifacts.
> Can this be solved by simply passing in the path to the artifacts? As well
> as the image tag for the locally build base StateFun image. They could
> probably be environment variables.
>
> Cheers,
> Gordon
>
> On Fri, Aug 18, 2023 at 12:13 PM Galen Warren via user <
> user@flink.apache.org> wrote:
>
>> Yes, exactly! And in addition to the base Statefun jars and the jar for
>> the Java SDK, it does an equivalent copy/register operation for each of the
>> other SDK libraries (Go, Python, Javascript) so that those libraries are
>> also available when building the playground examples.
>>
>> One more question: In order to copy the various build artifacts into the
>> Docker containers, those artifacts need to be part of the Docker context.
>> With the playground being a separate project, that's slightly tricky to do,
>> as there is no guarantee (other than convention) about the relative paths
>> of *flink-statefun* and* flink-statefun-playground *in someone's local
>> filesystem. The way I've set this up locally is to copy the playground into
>> the* flink-statefun* project -- i.e. to *flink-statefun*/playground --
>> such that I can set the Docker context to the root folder of
>> *flink-statefun* and then have access to any local code and/or build
>> artifacts.
>>
>> I'm wondering if there might be any appetite for making that move
>> permanent, i.e. moving the playground to *flink-statefun*/playground and
>> deprecating the standalone playground project. In addition to making the
>> problem of building with unreleased artifacts a bit simpler to solve, it
>> would also simplify the process of releasing a new Statefun version, since
>> the entire process could be handled with a single PR and associated
>> build/deploy tasks. In other words, a single PR could both update and
>> deploy the Statefun package and the playground code and images.
>>
>> As it stands, at least two PRs would be required for each Statefun
>> version update -- one for *flink-statefun* and one for
>> *flink-statefun-playground*.
>>
>> Anyway, just an idea. Maybe there's an important reason for these
>> projects to remain separate. If we do want to keep the playground project
>> where it is, I could solve the copying problem by requiring the two
>> projects to be siblings in the file system and by pre-copying the local
>> build artifacts into a location accessible by the existing Docker contexts.
>> This would still leave us with the need to have two PRs and releases
>> instead of one, though.
>>
>> Thanks for your help!
>>
>>
>> On Fri, Aug 18, 2023 at 2:45 PM Tzu-Li (Gordon) Tai <tz...@apache.org>
>> wrote:
>>
>>> Hi Galen,
>>>
>>> > locally built code is copied into the build containers
>>> so that it can be accessed during the build.
>>>
>>> That's exactly what we had been doing for release testing, yes. Sorry I
>>> missed that detail in my previous response.
>>>
>>> And yes, that sounds like a reasonable approach. If I understand you
>>> correctly, the workflow would become this:
>>>
>>>    1. Build the StateFun repo locally to install the snapshot artifact
>>>    jars + have a local base StateFun image.
>>>    2. Run the playground in "local" mode, so that it uses the local base
>>>    StateFun image + builds the playground code using copied artifact jars
>>>    (instead of pulling from Maven).
>>>
>>> That looks good to me!
>>>
>>> Thanks,
>>> Gordon
>>>
>>> On Fri, Aug 18, 2023 at 11:33 AM Galen Warren
>>> <ga...@cvillewarrens.com.invalid> wrote:
>>>
>>> > Thanks.
>>> >
>>> > If you were to build a local image, as you suggest, how do you access
>>> that
>>> > image when building the playground images? All the compilation of
>>> > playground code happens inside containers, so local images on the host
>>> > aren't available in those containers. Unless I'm missing something?
>>> >
>>> > I've slightly reworked things such that the playground images can be
>>> run in
>>> > one of two modes -- the default mode, which works like before, and a
>>> > "local" mode where locally built code is copied into the build
>>> containers
>>> > so that it can be accessed during the build. It works fine, you just
>>> have
>>> > to define a couple of environment variables when running
>>> docker-compose to
>>> > specify default vs. local mode and what versions of Flink and Statefun
>>> > should be referenced, and then you can build a run the local examples
>>> > without any additional steps. Does that sound like a reasonable
>>> approach?
>>> >
>>> >
>>> > On Fri, Aug 18, 2023 at 2:17 PM Tzu-Li (Gordon) Tai <
>>> tzulitai@apache.org>
>>> > wrote:
>>> >
>>> > > Hi Galen,
>>> > >
>>> > > > Gordon, is there a trick to running the sample code in
>>> > > flink-statefun-playground against yet-unreleased code that I'm
>>> missing?
>>> > >
>>> > > You'd have to locally build an image from the release branch, with a
>>> > > temporary image version tag. Then, in the flink-statefun-playground,
>>> > change
>>> > > the image versions in the docker-compose files to use that locally
>>> built
>>> > > image. IIRC, that's what we have been doing in the past. Admittedly,
>>> it's
>>> > > pretty manual - I don't think the CI manages this workflow.
>>> > >
>>> > > Thanks,
>>> > > Gordon
>>> > >
>>> > > On Mon, Aug 14, 2023 at 10:42 AM Galen Warren <
>>> galen@cvillewarrens.com>
>>> > > wrote:
>>> > >
>>> > > > I created a pull request for this: [FLINK-31619] Upgrade Stateful
>>> > > > Functions to Flink 1.16.1 by galenwarren · Pull Request #331 ·
>>> > > > apache/flink-statefun (github.com)
>>> > > > <https://github.com/apache/flink-statefun/pull/331>.
>>> > > >
>>> > > > JIRA is here: [FLINK-31619] Upgrade Stateful Functions to Flink
>>> 1.16.1
>>> > -
>>> > > > ASF JIRA (apache.org)
>>> > > > <https://issues.apache.org/jira/browse/FLINK-31619?filter=-1>.
>>> > > >
>>> > > > Statefun references 1.16.2, despite the title -- that version has
>>> come
>>> > > out
>>> > > > since the issue was created.
>>> > > >
>>> > > > I figured out how to run all the playground tests locally, but it
>>> took
>>> > a
>>> > > > bit of reworking of the playground setup with respect to Docker;
>>> > > > specifically, the Docker contexts used to build the example
>>> functions
>>> > > > needed to be broadened (i.e. moved up the tree) so that, if needed,
>>> > local
>>> > > > artifacts/code can be accessed from within the containers at build
>>> > time.
>>> > > > Then I made the Docker compose.yml configurable through environment
>>> > > > variables to allow for them to run in either the original manner --
>>> > i.e.
>>> > > > pulling artifacts from public repos -- or in a "local" mode, where
>>> > > > artifacts are pulled from local builds.
>>> > > >
>>> > > > This process is a cleaner if the playground is a subfolder of the
>>> > > > flink-statefun project rather than be its own repository
>>> > > > (flink-statefun-playground), because then all the relative paths
>>> > between
>>> > > > the playground files and the build artifacts are fixed. So, I'd
>>> like to
>>> > > > propose to move the playground files, modified as described above,
>>> to
>>> > > > flink-statefun/playground and retire flink-statefun-playground. I
>>> can
>>> > > > submit separate PR s those changes if everyone is on board.
>>> > > >
>>> > > > Also, should I plan to do the same upgrade to handle Flink 1.17.x?
>>> It
>>> > > > should be easy to do, especially while the 1.16.x upgrade is fresh
>>> on
>>> > my
>>> > > > mind.
>>> > > >
>>> > > > Thanks.
>>> > > >
>>> > > >
>>> > > > On Fri, Aug 11, 2023 at 6:40 PM Galen Warren <
>>> galen@cvillewarrens.com>
>>> > > > wrote:
>>> > > >
>>> > > >> I'm done with the code to make Statefun compatible with Flink
>>> 1.16,
>>> > and
>>> > > >> all the tests (including e2e succeed). The required changes were
>>> > pretty
>>> > > >> minimal.
>>> > > >>
>>> > > >> I'm running into a bit of a chicken/egg problem executing the
>>> tests in
>>> > > >> flink-statefun-playground
>>> > > >> <https://github.com/apache/flink-statefun-playground>, though.
>>> That
>>> > > >> project seems to assume that all the various Statefun artifacts
>>> are
>>> > > built
>>> > > >> and deployed to the various public repositories already. I've
>>> looked
>>> > > into
>>> > > >> trying to redirect references to local artifacts; however, that's
>>> also
>>> > > >> tricky since all the building occurs in Docker containers.
>>> > > >>
>>> > > >> Gordon, is there a trick to running the sample code in
>>> > > >> flink-statefun-playground against yet-unreleased code that I'm
>>> > missing?
>>> > > >>
>>> > > >> Thanks.
>>> > > >>
>>> > > >> On Sat, Jun 24, 2023 at 12:40 PM Galen Warren <
>>> > galen@cvillewarrens.com>
>>> > > >> wrote:
>>> > > >>
>>> > > >>> Great -- thanks!
>>> > > >>>
>>> > > >>> I'm going to be out of town for about a week but I'll take a
>>> look at
>>> > > >>> this when I'm back.
>>> > > >>>
>>> > > >>> On Tue, Jun 20, 2023 at 8:46 AM Martijn Visser <
>>> mvisser@confluent.io
>>> > >
>>> > > >>> wrote:
>>> > > >>>
>>> > > >>>> Hi Galen,
>>> > > >>>>
>>> > > >>>> Yes, I'll be more than happy to help with Statefun releases.
>>> > > >>>>
>>> > > >>>> Best regards,
>>> > > >>>>
>>> > > >>>> Martijn
>>> > > >>>>
>>> > > >>>> On Tue, Jun 20, 2023 at 2:21 PM Galen Warren <
>>> > galen@cvillewarrens.com
>>> > > >
>>> > > >>>> wrote:
>>> > > >>>>
>>> > > >>>>> Thanks.
>>> > > >>>>>
>>> > > >>>>> Martijn, to answer your question, I'd need to do a small
>>> amount of
>>> > > >>>>> work to get a PR ready, but not much. Happy to do it if we're
>>> > > deciding to
>>> > > >>>>> restart Statefun releases -- are we?
>>> > > >>>>>
>>> > > >>>>> -- Galen
>>> > > >>>>>
>>> > > >>>>> On Sat, Jun 17, 2023 at 9:47 AM Tzu-Li (Gordon) Tai <
>>> > > >>>>> tzulitai@apache.org> wrote:
>>> > > >>>>>
>>> > > >>>>>> > Perhaps he could weigh in on whether the combination of
>>> > automated
>>> > > >>>>>> tests plus those smoke tests should be sufficient for testing
>>> with
>>> > > new
>>> > > >>>>>> Flink versions
>>> > > >>>>>>
>>> > > >>>>>> What we usually did at the bare minimum for new StateFun
>>> releases
>>> > > was
>>> > > >>>>>> the following:
>>> > > >>>>>>
>>> > > >>>>>>    1. Build tests (including the smoke tests in the e2e
>>> module,
>>> > > >>>>>>    which covers important tests like exactly-once
>>> verification)
>>> > > >>>>>>    2. Updating the flink-statefun-playground repo and manually
>>> > > >>>>>>    running all language examples there.
>>> > > >>>>>>
>>> > > >>>>>> If upgrading Flink versions was the only change in the
>>> release,
>>> > I'd
>>> > > >>>>>> probably say that this is sufficient.
>>> > > >>>>>>
>>> > > >>>>>> Best,
>>> > > >>>>>> Gordon
>>> > > >>>>>>
>>> > > >>>>>> On Thu, Jun 15, 2023 at 5:25 AM Martijn Visser <
>>> > > >>>>>> martijnvisser@apache.org> wrote:
>>> > > >>>>>>
>>> > > >>>>>>> Let me know if you have a PR for a Flink update :)
>>> > > >>>>>>>
>>> > > >>>>>>> On Thu, Jun 8, 2023 at 5:52 PM Galen Warren via user <
>>> > > >>>>>>> user@flink.apache.org> wrote:
>>> > > >>>>>>>
>>> > > >>>>>>>> Thanks Martijn.
>>> > > >>>>>>>>
>>> > > >>>>>>>> Personally, I'm already using a local fork of Statefun that
>>> is
>>> > > >>>>>>>> compatible with Flink 1.16.x, so I wouldn't have any need
>>> for a
>>> > > released
>>> > > >>>>>>>> version compatible with 1.15.x. I'd be happy to do the PRs
>>> to
>>> > > modify
>>> > > >>>>>>>> Statefun to work with new versions of Flink as they come
>>> along.
>>> > > >>>>>>>>
>>> > > >>>>>>>> As for testing, Statefun does have unit tests and Gordon
>>> also
>>> > sent
>>> > > >>>>>>>> me instructions a while back for how to do some additional
>>> smoke
>>> > > tests
>>> > > >>>>>>>> which are pretty straightforward. Perhaps he could weigh in
>>> on
>>> > > whether the
>>> > > >>>>>>>> combination of automated tests plus those smoke tests
>>> should be
>>> > > sufficient
>>> > > >>>>>>>> for testing with new Flink versions (I believe the answer is
>>> > yes).
>>> > > >>>>>>>>
>>> > > >>>>>>>> -- Galen
>>> > > >>>>>>>>
>>> > > >>>>>>>>
>>> > > >>>>>>>>
>>> > > >>>>>>>> On Thu, Jun 8, 2023 at 8:01 AM Martijn Visser <
>>> > > >>>>>>>> martijnvisser@apache.org> wrote:
>>> > > >>>>>>>>
>>> > > >>>>>>>>> Hi all,
>>> > > >>>>>>>>>
>>> > > >>>>>>>>> Apologies for the late reply.
>>> > > >>>>>>>>>
>>> > > >>>>>>>>> I'm willing to help out with merging requests in Statefun
>>> to
>>> > keep
>>> > > >>>>>>>>> them
>>> > > >>>>>>>>> compatible with new Flink releases and create new
>>> releases. I
>>> > do
>>> > > >>>>>>>>> think that
>>> > > >>>>>>>>> validation of the functionality of these releases depends
>>> a lot
>>> > > on
>>> > > >>>>>>>>> those
>>> > > >>>>>>>>> who do these compatibility updates, with PMC members
>>> helping
>>> > out
>>> > > >>>>>>>>> with the
>>> > > >>>>>>>>> formal process.
>>> > > >>>>>>>>>
>>> > > >>>>>>>>> > Why can't the Apache Software Foundation allow community
>>> > > members
>>> > > >>>>>>>>> to bring
>>> > > >>>>>>>>> it up to date?
>>> > > >>>>>>>>>
>>> > > >>>>>>>>> There's nothing preventing anyone from reviewing any of the
>>> > > >>>>>>>>> current PRs or
>>> > > >>>>>>>>> opening new ones. However, none of them are approved [1],
>>> so
>>> > > >>>>>>>>> there's also
>>> > > >>>>>>>>> nothing to merge.
>>> > > >>>>>>>>>
>>> > > >>>>>>>>> > I believe that there are people and companies on this
>>> mailing
>>> > > >>>>>>>>> list
>>> > > >>>>>>>>> interested in supporting Apache Flink Stateful Functions.
>>> > > >>>>>>>>>
>>> > > >>>>>>>>> If so, then now is the time to show.
>>> > > >>>>>>>>>
>>> > > >>>>>>>>> Would there be a preference to create a release with
>>> Galen's
>>> > > merged
>>> > > >>>>>>>>> compatibility update to Flink 1.15.2, or do we want to skip
>>> > that
>>> > > >>>>>>>>> and go
>>> > > >>>>>>>>> straight to a newer version?
>>> > > >>>>>>>>>
>>> > > >>>>>>>>> Best regards,
>>> > > >>>>>>>>>
>>> > > >>>>>>>>> Martijn
>>> > > >>>>>>>>>
>>> > > >>>>>>>>> [1]
>>> > > >>>>>>>>>
>>> > > >>>>>>>>>
>>> > >
>>> >
>>> https://github.com/apache/flink-statefun/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved
>>> > > >>>>>>>>>
>>> > > >>>>>>>>> On Tue, Jun 6, 2023 at 3:55 PM Marco Villalobos <
>>> > > >>>>>>>>> mvillalobos@kineteque.com>
>>> > > >>>>>>>>> wrote:
>>> > > >>>>>>>>>
>>> > > >>>>>>>>> > Why can't the Apache Software Foundation allow community
>>> > > members
>>> > > >>>>>>>>> to bring
>>> > > >>>>>>>>> > it up to date?
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > What's the process for that?
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > I believe that there are people and companies on this
>>> mailing
>>> > > >>>>>>>>> list
>>> > > >>>>>>>>> > interested in supporting Apache Flink Stateful Functions.
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > You already had two people on this thread express
>>> interest.
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > At the very least, we could keep the library versions up
>>> to
>>> > > date.
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > There are only a small list of new features that might be
>>> > > >>>>>>>>> worthwhile:
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > 1. event time processing
>>> > > >>>>>>>>> > 2. state rest api
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <
>>> > > chesnay@apache.org>
>>> > > >>>>>>>>> wrote:
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > If you were to fork it *and want to redistribute it*
>>> then the
>>> > > >>>>>>>>> short
>>> > > >>>>>>>>> > version is that
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> >    1. you have to adhere to the Apache licensing
>>> requirements
>>> > > >>>>>>>>> >    2. you have to make it clear that your fork does not
>>> > belong
>>> > > >>>>>>>>> to the
>>> > > >>>>>>>>> >    Apache Flink project. (Trademarks and all that)
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > Neither should be significant hurdles (there should also
>>> be
>>> > > >>>>>>>>> plenty of
>>> > > >>>>>>>>> > online resources regarding 1), and if you do this then
>>> you
>>> > can
>>> > > >>>>>>>>> freely share
>>> > > >>>>>>>>> > your fork with others.
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > I've also pinged Martijn to take a look at this thread.
>>> > > >>>>>>>>> > To my knowledge the project hasn't decided anything yet.
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > On 27/05/2023 04:05, Galen Warren wrote:
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > Ok, I get it. No interest.
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > If this project is being abandoned, I guess I'll work
>>> with my
>>> > > >>>>>>>>> own fork. Is
>>> > > >>>>>>>>> > there anything I should consider here? Can I share it
>>> with
>>> > > other
>>> > > >>>>>>>>> people who
>>> > > >>>>>>>>> > use this project?
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > On Tue, May 16, 2023 at 10:50 AM Galen Warren <
>>> > > >>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
>>> > > >>>>>>>>> > wrote:
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > Hi Martijn, since you opened this discussion thread, I'm
>>> > > curious
>>> > > >>>>>>>>> what your
>>> > > >>>>>>>>> > thoughts are in light of the responses? Thanks.
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <
>>> > > >>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
>>> > > >>>>>>>>> > wrote:
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > I use Apache Flink for stream processing, and StateFun
>>> as a
>>> > > >>>>>>>>> hand-off
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > point for the rest of the application.
>>> > > >>>>>>>>> > It serves well as a bridge between a Flink Streaming job
>>> and
>>> > > >>>>>>>>> > micro-services.
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > This is essentially how I use it as well, and I would
>>> also be
>>> > > >>>>>>>>> sad to see
>>> > > >>>>>>>>> > it sunsetted. It works well; I don't know that there is
>>> a lot
>>> > > of
>>> > > >>>>>>>>> new
>>> > > >>>>>>>>> > development required, but if there are no new Statefun
>>> > > releases,
>>> > > >>>>>>>>> then
>>> > > >>>>>>>>> > Statefun can only be used with older Flink versions.
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
>>> > > >>>>>>>>> mvillalobos@kineteque.com> wrote:
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > I am currently using Stateful Functions in my
>>> application.
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > I use Apache Flink for stream processing, and StateFun
>>> as a
>>> > > >>>>>>>>> hand-off
>>> > > >>>>>>>>> > point for the rest of the application.
>>> > > >>>>>>>>> > It serves well as a bridge between a Flink Streaming job
>>> and
>>> > > >>>>>>>>> > micro-services.
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > I would be disappointed if StateFun was sunsetted.  Its a
>>> > good
>>> > > >>>>>>>>> idea.
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > If there is anything I can do to help, as a contributor
>>> > > perhaps,
>>> > > >>>>>>>>> please
>>> > > >>>>>>>>> > let me know.
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <
>>> > > >>>>>>>>> martijnvisser@apache.org> <ma...@apache.org>
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > wrote:
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > Hi everyone,
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > I want to open a discussion on the status of the Statefun
>>> > > >>>>>>>>> Project [1]
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > in Apache Flink. As you might have noticed, there hasn't
>>> been
>>> > > >>>>>>>>> much
>>> > > >>>>>>>>> > development over the past months in the Statefun
>>> repository
>>> > > [2].
>>> > > >>>>>>>>> There is
>>> > > >>>>>>>>> > currently a lack of active contributors and committers
>>> who
>>> > are
>>> > > >>>>>>>>> able to help
>>> > > >>>>>>>>> > with the maintenance of the project.
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > In order to improve the situation, we need to solve the
>>> lack
>>> > of
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > committers and the lack of contributors.
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > On the lack of committers:
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > 1. Ideally, there are some of the current Flink
>>> committers
>>> > who
>>> > > >>>>>>>>> have
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > the bandwidth and can help with reviewing PRs and merging
>>> > them.
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > 2. If that's not an option, it could be a consideration
>>> that
>>> > > >>>>>>>>> current
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > committers only approve and review PRs, that are
>>> approved by
>>> > > >>>>>>>>> those who are
>>> > > >>>>>>>>> > willing to contribute to Statefun and if the CI passes
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > On the lack of contributors:
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > 3. Next to having this discussion on the Dev and User
>>> mailing
>>> > > >>>>>>>>> list, we
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > can also create a blog with a call for new contributors
>>> on
>>> > the
>>> > > >>>>>>>>> Flink
>>> > > >>>>>>>>> > project website, send out some tweets on the Flink /
>>> Statefun
>>> > > >>>>>>>>> twitter
>>> > > >>>>>>>>> > accounts, post messages on Slack etc. In that message, we
>>> > would
>>> > > >>>>>>>>> inform how
>>> > > >>>>>>>>> > those that are interested in contributing can start and
>>> where
>>> > > >>>>>>>>> they could
>>> > > >>>>>>>>> > reach out for more information.
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > There's also option 4. where a group of interested people
>>> > would
>>> > > >>>>>>>>> split
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > Statefun from the Flink project and make it a separate
>>> top
>>> > > level
>>> > > >>>>>>>>> project
>>> > > >>>>>>>>> > under the Apache Flink umbrella (similar as recently has
>>> > > >>>>>>>>> happened with
>>> > > >>>>>>>>> > Flink Table Store, which has become Apache Paimon).
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > If we see no improvements in the coming period, we should
>>> > > >>>>>>>>> consider
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > sunsetting Statefun and communicate that clearly to the
>>> > users.
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > I'm looking forward to your thoughts.
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > Best regards,
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > Martijn
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > [1]
>>> > > >>>>>>>>>
>>> https://nightlies.apache.org/flink/flink-statefun-docs-master/
>>> > <
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> >
>>> > https://nightlies.apache.org/flink/flink-statefun-docs-master/
>>> > > >
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > [2] https://github.com/apache/flink-statefun <
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > https://github.com/apache/flink-statefun>
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>>
>>> > > >>>>>>>>
>>> > >
>>> >
>>>
>>

Re: [DISCUSS] Status of Statefun Project

[Galen Warren via user <us...@flink.apache.org>]

Gotcha, makes sense as to the original division.

>> Can this be solved by simply passing in the path to the artifacts

This definitely works if we're going to be copying the artifacts on the
host side -- into the build context -- and then from the context into the
image. It only gets tricky to have a potentially varying path to the
artifacts if we're trying to *directly *include the artifacts in the Docker
context -- then we have a situation where the Docker context must contain
both the artifacts and playground files, with (potentially) different root
locations.

Maybe the simplest thing to do here is just to leave the playground as-is
and then copy the artifacts into the Docker context manually, prior to
building the playground images. I'm fine with that. It will mean that each
Statefun release will require two PRs and two sets of build/publish steps
instead of one, but if everyone else is fine with that I am, too. Unless
anyone objects, I'll go ahead and queue up a PR for the playground that
makes these changes.

Also, I should mention -- in case it's not clear -- that I have already
built and run the playground examples with the code from the PR and
everything worked. So that PR is ready to move forward with review, etc.,
at this point.

Thanks.







On Fri, Aug 18, 2023 at 4:16 PM Tzu-Li (Gordon) Tai <tz...@apache.org>
wrote:

> Hi Galen,
>
> The original intent of having a separate repo for the playground repo, was
> that StateFun users can just go to that and start running simple examples
> without any other distractions from the core code. I personally don't have
> a strong preference here and can understand how it would make the workflow
> more streamlined, but just FYI on the reasoning why are separate in the
> first place.
>
> re: paths for locating StateFun artifacts.
> Can this be solved by simply passing in the path to the artifacts? As well
> as the image tag for the locally build base StateFun image. They could
> probably be environment variables.
>
> Cheers,
> Gordon
>
> On Fri, Aug 18, 2023 at 12:13 PM Galen Warren via user <
> user@flink.apache.org> wrote:
>
>> Yes, exactly! And in addition to the base Statefun jars and the jar for
>> the Java SDK, it does an equivalent copy/register operation for each of the
>> other SDK libraries (Go, Python, Javascript) so that those libraries are
>> also available when building the playground examples.
>>
>> One more question: In order to copy the various build artifacts into the
>> Docker containers, those artifacts need to be part of the Docker context.
>> With the playground being a separate project, that's slightly tricky to do,
>> as there is no guarantee (other than convention) about the relative paths
>> of *flink-statefun* and* flink-statefun-playground *in someone's local
>> filesystem. The way I've set this up locally is to copy the playground into
>> the* flink-statefun* project -- i.e. to *flink-statefun*/playground --
>> such that I can set the Docker context to the root folder of
>> *flink-statefun* and then have access to any local code and/or build
>> artifacts.
>>
>> I'm wondering if there might be any appetite for making that move
>> permanent, i.e. moving the playground to *flink-statefun*/playground and
>> deprecating the standalone playground project. In addition to making the
>> problem of building with unreleased artifacts a bit simpler to solve, it
>> would also simplify the process of releasing a new Statefun version, since
>> the entire process could be handled with a single PR and associated
>> build/deploy tasks. In other words, a single PR could both update and
>> deploy the Statefun package and the playground code and images.
>>
>> As it stands, at least two PRs would be required for each Statefun
>> version update -- one for *flink-statefun* and one for
>> *flink-statefun-playground*.
>>
>> Anyway, just an idea. Maybe there's an important reason for these
>> projects to remain separate. If we do want to keep the playground project
>> where it is, I could solve the copying problem by requiring the two
>> projects to be siblings in the file system and by pre-copying the local
>> build artifacts into a location accessible by the existing Docker contexts.
>> This would still leave us with the need to have two PRs and releases
>> instead of one, though.
>>
>> Thanks for your help!
>>
>>
>> On Fri, Aug 18, 2023 at 2:45 PM Tzu-Li (Gordon) Tai <tz...@apache.org>
>> wrote:
>>
>>> Hi Galen,
>>>
>>> > locally built code is copied into the build containers
>>> so that it can be accessed during the build.
>>>
>>> That's exactly what we had been doing for release testing, yes. Sorry I
>>> missed that detail in my previous response.
>>>
>>> And yes, that sounds like a reasonable approach. If I understand you
>>> correctly, the workflow would become this:
>>>
>>>    1. Build the StateFun repo locally to install the snapshot artifact
>>>    jars + have a local base StateFun image.
>>>    2. Run the playground in "local" mode, so that it uses the local base
>>>    StateFun image + builds the playground code using copied artifact jars
>>>    (instead of pulling from Maven).
>>>
>>> That looks good to me!
>>>
>>> Thanks,
>>> Gordon
>>>
>>> On Fri, Aug 18, 2023 at 11:33 AM Galen Warren
>>> <ga...@cvillewarrens.com.invalid> wrote:
>>>
>>> > Thanks.
>>> >
>>> > If you were to build a local image, as you suggest, how do you access
>>> that
>>> > image when building the playground images? All the compilation of
>>> > playground code happens inside containers, so local images on the host
>>> > aren't available in those containers. Unless I'm missing something?
>>> >
>>> > I've slightly reworked things such that the playground images can be
>>> run in
>>> > one of two modes -- the default mode, which works like before, and a
>>> > "local" mode where locally built code is copied into the build
>>> containers
>>> > so that it can be accessed during the build. It works fine, you just
>>> have
>>> > to define a couple of environment variables when running
>>> docker-compose to
>>> > specify default vs. local mode and what versions of Flink and Statefun
>>> > should be referenced, and then you can build a run the local examples
>>> > without any additional steps. Does that sound like a reasonable
>>> approach?
>>> >
>>> >
>>> > On Fri, Aug 18, 2023 at 2:17 PM Tzu-Li (Gordon) Tai <
>>> tzulitai@apache.org>
>>> > wrote:
>>> >
>>> > > Hi Galen,
>>> > >
>>> > > > Gordon, is there a trick to running the sample code in
>>> > > flink-statefun-playground against yet-unreleased code that I'm
>>> missing?
>>> > >
>>> > > You'd have to locally build an image from the release branch, with a
>>> > > temporary image version tag. Then, in the flink-statefun-playground,
>>> > change
>>> > > the image versions in the docker-compose files to use that locally
>>> built
>>> > > image. IIRC, that's what we have been doing in the past. Admittedly,
>>> it's
>>> > > pretty manual - I don't think the CI manages this workflow.
>>> > >
>>> > > Thanks,
>>> > > Gordon
>>> > >
>>> > > On Mon, Aug 14, 2023 at 10:42 AM Galen Warren <
>>> galen@cvillewarrens.com>
>>> > > wrote:
>>> > >
>>> > > > I created a pull request for this: [FLINK-31619] Upgrade Stateful
>>> > > > Functions to Flink 1.16.1 by galenwarren · Pull Request #331 ·
>>> > > > apache/flink-statefun (github.com)
>>> > > > <https://github.com/apache/flink-statefun/pull/331>.
>>> > > >
>>> > > > JIRA is here: [FLINK-31619] Upgrade Stateful Functions to Flink
>>> 1.16.1
>>> > -
>>> > > > ASF JIRA (apache.org)
>>> > > > <https://issues.apache.org/jira/browse/FLINK-31619?filter=-1>.
>>> > > >
>>> > > > Statefun references 1.16.2, despite the title -- that version has
>>> come
>>> > > out
>>> > > > since the issue was created.
>>> > > >
>>> > > > I figured out how to run all the playground tests locally, but it
>>> took
>>> > a
>>> > > > bit of reworking of the playground setup with respect to Docker;
>>> > > > specifically, the Docker contexts used to build the example
>>> functions
>>> > > > needed to be broadened (i.e. moved up the tree) so that, if needed,
>>> > local
>>> > > > artifacts/code can be accessed from within the containers at build
>>> > time.
>>> > > > Then I made the Docker compose.yml configurable through environment
>>> > > > variables to allow for them to run in either the original manner --
>>> > i.e.
>>> > > > pulling artifacts from public repos -- or in a "local" mode, where
>>> > > > artifacts are pulled from local builds.
>>> > > >
>>> > > > This process is a cleaner if the playground is a subfolder of the
>>> > > > flink-statefun project rather than be its own repository
>>> > > > (flink-statefun-playground), because then all the relative paths
>>> > between
>>> > > > the playground files and the build artifacts are fixed. So, I'd
>>> like to
>>> > > > propose to move the playground files, modified as described above,
>>> to
>>> > > > flink-statefun/playground and retire flink-statefun-playground. I
>>> can
>>> > > > submit separate PR s those changes if everyone is on board.
>>> > > >
>>> > > > Also, should I plan to do the same upgrade to handle Flink 1.17.x?
>>> It
>>> > > > should be easy to do, especially while the 1.16.x upgrade is fresh
>>> on
>>> > my
>>> > > > mind.
>>> > > >
>>> > > > Thanks.
>>> > > >
>>> > > >
>>> > > > On Fri, Aug 11, 2023 at 6:40 PM Galen Warren <
>>> galen@cvillewarrens.com>
>>> > > > wrote:
>>> > > >
>>> > > >> I'm done with the code to make Statefun compatible with Flink
>>> 1.16,
>>> > and
>>> > > >> all the tests (including e2e succeed). The required changes were
>>> > pretty
>>> > > >> minimal.
>>> > > >>
>>> > > >> I'm running into a bit of a chicken/egg problem executing the
>>> tests in
>>> > > >> flink-statefun-playground
>>> > > >> <https://github.com/apache/flink-statefun-playground>, though.
>>> That
>>> > > >> project seems to assume that all the various Statefun artifacts
>>> are
>>> > > built
>>> > > >> and deployed to the various public repositories already. I've
>>> looked
>>> > > into
>>> > > >> trying to redirect references to local artifacts; however, that's
>>> also
>>> > > >> tricky since all the building occurs in Docker containers.
>>> > > >>
>>> > > >> Gordon, is there a trick to running the sample code in
>>> > > >> flink-statefun-playground against yet-unreleased code that I'm
>>> > missing?
>>> > > >>
>>> > > >> Thanks.
>>> > > >>
>>> > > >> On Sat, Jun 24, 2023 at 12:40 PM Galen Warren <
>>> > galen@cvillewarrens.com>
>>> > > >> wrote:
>>> > > >>
>>> > > >>> Great -- thanks!
>>> > > >>>
>>> > > >>> I'm going to be out of town for about a week but I'll take a
>>> look at
>>> > > >>> this when I'm back.
>>> > > >>>
>>> > > >>> On Tue, Jun 20, 2023 at 8:46 AM Martijn Visser <
>>> mvisser@confluent.io
>>> > >
>>> > > >>> wrote:
>>> > > >>>
>>> > > >>>> Hi Galen,
>>> > > >>>>
>>> > > >>>> Yes, I'll be more than happy to help with Statefun releases.
>>> > > >>>>
>>> > > >>>> Best regards,
>>> > > >>>>
>>> > > >>>> Martijn
>>> > > >>>>
>>> > > >>>> On Tue, Jun 20, 2023 at 2:21 PM Galen Warren <
>>> > galen@cvillewarrens.com
>>> > > >
>>> > > >>>> wrote:
>>> > > >>>>
>>> > > >>>>> Thanks.
>>> > > >>>>>
>>> > > >>>>> Martijn, to answer your question, I'd need to do a small
>>> amount of
>>> > > >>>>> work to get a PR ready, but not much. Happy to do it if we're
>>> > > deciding to
>>> > > >>>>> restart Statefun releases -- are we?
>>> > > >>>>>
>>> > > >>>>> -- Galen
>>> > > >>>>>
>>> > > >>>>> On Sat, Jun 17, 2023 at 9:47 AM Tzu-Li (Gordon) Tai <
>>> > > >>>>> tzulitai@apache.org> wrote:
>>> > > >>>>>
>>> > > >>>>>> > Perhaps he could weigh in on whether the combination of
>>> > automated
>>> > > >>>>>> tests plus those smoke tests should be sufficient for testing
>>> with
>>> > > new
>>> > > >>>>>> Flink versions
>>> > > >>>>>>
>>> > > >>>>>> What we usually did at the bare minimum for new StateFun
>>> releases
>>> > > was
>>> > > >>>>>> the following:
>>> > > >>>>>>
>>> > > >>>>>>    1. Build tests (including the smoke tests in the e2e
>>> module,
>>> > > >>>>>>    which covers important tests like exactly-once
>>> verification)
>>> > > >>>>>>    2. Updating the flink-statefun-playground repo and manually
>>> > > >>>>>>    running all language examples there.
>>> > > >>>>>>
>>> > > >>>>>> If upgrading Flink versions was the only change in the
>>> release,
>>> > I'd
>>> > > >>>>>> probably say that this is sufficient.
>>> > > >>>>>>
>>> > > >>>>>> Best,
>>> > > >>>>>> Gordon
>>> > > >>>>>>
>>> > > >>>>>> On Thu, Jun 15, 2023 at 5:25 AM Martijn Visser <
>>> > > >>>>>> martijnvisser@apache.org> wrote:
>>> > > >>>>>>
>>> > > >>>>>>> Let me know if you have a PR for a Flink update :)
>>> > > >>>>>>>
>>> > > >>>>>>> On Thu, Jun 8, 2023 at 5:52 PM Galen Warren via user <
>>> > > >>>>>>> user@flink.apache.org> wrote:
>>> > > >>>>>>>
>>> > > >>>>>>>> Thanks Martijn.
>>> > > >>>>>>>>
>>> > > >>>>>>>> Personally, I'm already using a local fork of Statefun that
>>> is
>>> > > >>>>>>>> compatible with Flink 1.16.x, so I wouldn't have any need
>>> for a
>>> > > released
>>> > > >>>>>>>> version compatible with 1.15.x. I'd be happy to do the PRs
>>> to
>>> > > modify
>>> > > >>>>>>>> Statefun to work with new versions of Flink as they come
>>> along.
>>> > > >>>>>>>>
>>> > > >>>>>>>> As for testing, Statefun does have unit tests and Gordon
>>> also
>>> > sent
>>> > > >>>>>>>> me instructions a while back for how to do some additional
>>> smoke
>>> > > tests
>>> > > >>>>>>>> which are pretty straightforward. Perhaps he could weigh in
>>> on
>>> > > whether the
>>> > > >>>>>>>> combination of automated tests plus those smoke tests
>>> should be
>>> > > sufficient
>>> > > >>>>>>>> for testing with new Flink versions (I believe the answer is
>>> > yes).
>>> > > >>>>>>>>
>>> > > >>>>>>>> -- Galen
>>> > > >>>>>>>>
>>> > > >>>>>>>>
>>> > > >>>>>>>>
>>> > > >>>>>>>> On Thu, Jun 8, 2023 at 8:01 AM Martijn Visser <
>>> > > >>>>>>>> martijnvisser@apache.org> wrote:
>>> > > >>>>>>>>
>>> > > >>>>>>>>> Hi all,
>>> > > >>>>>>>>>
>>> > > >>>>>>>>> Apologies for the late reply.
>>> > > >>>>>>>>>
>>> > > >>>>>>>>> I'm willing to help out with merging requests in Statefun
>>> to
>>> > keep
>>> > > >>>>>>>>> them
>>> > > >>>>>>>>> compatible with new Flink releases and create new
>>> releases. I
>>> > do
>>> > > >>>>>>>>> think that
>>> > > >>>>>>>>> validation of the functionality of these releases depends
>>> a lot
>>> > > on
>>> > > >>>>>>>>> those
>>> > > >>>>>>>>> who do these compatibility updates, with PMC members
>>> helping
>>> > out
>>> > > >>>>>>>>> with the
>>> > > >>>>>>>>> formal process.
>>> > > >>>>>>>>>
>>> > > >>>>>>>>> > Why can't the Apache Software Foundation allow community
>>> > > members
>>> > > >>>>>>>>> to bring
>>> > > >>>>>>>>> it up to date?
>>> > > >>>>>>>>>
>>> > > >>>>>>>>> There's nothing preventing anyone from reviewing any of the
>>> > > >>>>>>>>> current PRs or
>>> > > >>>>>>>>> opening new ones. However, none of them are approved [1],
>>> so
>>> > > >>>>>>>>> there's also
>>> > > >>>>>>>>> nothing to merge.
>>> > > >>>>>>>>>
>>> > > >>>>>>>>> > I believe that there are people and companies on this
>>> mailing
>>> > > >>>>>>>>> list
>>> > > >>>>>>>>> interested in supporting Apache Flink Stateful Functions.
>>> > > >>>>>>>>>
>>> > > >>>>>>>>> If so, then now is the time to show.
>>> > > >>>>>>>>>
>>> > > >>>>>>>>> Would there be a preference to create a release with
>>> Galen's
>>> > > merged
>>> > > >>>>>>>>> compatibility update to Flink 1.15.2, or do we want to skip
>>> > that
>>> > > >>>>>>>>> and go
>>> > > >>>>>>>>> straight to a newer version?
>>> > > >>>>>>>>>
>>> > > >>>>>>>>> Best regards,
>>> > > >>>>>>>>>
>>> > > >>>>>>>>> Martijn
>>> > > >>>>>>>>>
>>> > > >>>>>>>>> [1]
>>> > > >>>>>>>>>
>>> > > >>>>>>>>>
>>> > >
>>> >
>>> https://github.com/apache/flink-statefun/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved
>>> > > >>>>>>>>>
>>> > > >>>>>>>>> On Tue, Jun 6, 2023 at 3:55 PM Marco Villalobos <
>>> > > >>>>>>>>> mvillalobos@kineteque.com>
>>> > > >>>>>>>>> wrote:
>>> > > >>>>>>>>>
>>> > > >>>>>>>>> > Why can't the Apache Software Foundation allow community
>>> > > members
>>> > > >>>>>>>>> to bring
>>> > > >>>>>>>>> > it up to date?
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > What's the process for that?
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > I believe that there are people and companies on this
>>> mailing
>>> > > >>>>>>>>> list
>>> > > >>>>>>>>> > interested in supporting Apache Flink Stateful Functions.
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > You already had two people on this thread express
>>> interest.
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > At the very least, we could keep the library versions up
>>> to
>>> > > date.
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > There are only a small list of new features that might be
>>> > > >>>>>>>>> worthwhile:
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > 1. event time processing
>>> > > >>>>>>>>> > 2. state rest api
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <
>>> > > chesnay@apache.org>
>>> > > >>>>>>>>> wrote:
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > If you were to fork it *and want to redistribute it*
>>> then the
>>> > > >>>>>>>>> short
>>> > > >>>>>>>>> > version is that
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> >    1. you have to adhere to the Apache licensing
>>> requirements
>>> > > >>>>>>>>> >    2. you have to make it clear that your fork does not
>>> > belong
>>> > > >>>>>>>>> to the
>>> > > >>>>>>>>> >    Apache Flink project. (Trademarks and all that)
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > Neither should be significant hurdles (there should also
>>> be
>>> > > >>>>>>>>> plenty of
>>> > > >>>>>>>>> > online resources regarding 1), and if you do this then
>>> you
>>> > can
>>> > > >>>>>>>>> freely share
>>> > > >>>>>>>>> > your fork with others.
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > I've also pinged Martijn to take a look at this thread.
>>> > > >>>>>>>>> > To my knowledge the project hasn't decided anything yet.
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > On 27/05/2023 04:05, Galen Warren wrote:
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > Ok, I get it. No interest.
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > If this project is being abandoned, I guess I'll work
>>> with my
>>> > > >>>>>>>>> own fork. Is
>>> > > >>>>>>>>> > there anything I should consider here? Can I share it
>>> with
>>> > > other
>>> > > >>>>>>>>> people who
>>> > > >>>>>>>>> > use this project?
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > On Tue, May 16, 2023 at 10:50 AM Galen Warren <
>>> > > >>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
>>> > > >>>>>>>>> > wrote:
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > Hi Martijn, since you opened this discussion thread, I'm
>>> > > curious
>>> > > >>>>>>>>> what your
>>> > > >>>>>>>>> > thoughts are in light of the responses? Thanks.
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <
>>> > > >>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
>>> > > >>>>>>>>> > wrote:
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > I use Apache Flink for stream processing, and StateFun
>>> as a
>>> > > >>>>>>>>> hand-off
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > point for the rest of the application.
>>> > > >>>>>>>>> > It serves well as a bridge between a Flink Streaming job
>>> and
>>> > > >>>>>>>>> > micro-services.
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > This is essentially how I use it as well, and I would
>>> also be
>>> > > >>>>>>>>> sad to see
>>> > > >>>>>>>>> > it sunsetted. It works well; I don't know that there is
>>> a lot
>>> > > of
>>> > > >>>>>>>>> new
>>> > > >>>>>>>>> > development required, but if there are no new Statefun
>>> > > releases,
>>> > > >>>>>>>>> then
>>> > > >>>>>>>>> > Statefun can only be used with older Flink versions.
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
>>> > > >>>>>>>>> mvillalobos@kineteque.com> wrote:
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > I am currently using Stateful Functions in my
>>> application.
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > I use Apache Flink for stream processing, and StateFun
>>> as a
>>> > > >>>>>>>>> hand-off
>>> > > >>>>>>>>> > point for the rest of the application.
>>> > > >>>>>>>>> > It serves well as a bridge between a Flink Streaming job
>>> and
>>> > > >>>>>>>>> > micro-services.
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > I would be disappointed if StateFun was sunsetted.  Its a
>>> > good
>>> > > >>>>>>>>> idea.
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > If there is anything I can do to help, as a contributor
>>> > > perhaps,
>>> > > >>>>>>>>> please
>>> > > >>>>>>>>> > let me know.
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <
>>> > > >>>>>>>>> martijnvisser@apache.org> <ma...@apache.org>
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > wrote:
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > Hi everyone,
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > I want to open a discussion on the status of the Statefun
>>> > > >>>>>>>>> Project [1]
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > in Apache Flink. As you might have noticed, there hasn't
>>> been
>>> > > >>>>>>>>> much
>>> > > >>>>>>>>> > development over the past months in the Statefun
>>> repository
>>> > > [2].
>>> > > >>>>>>>>> There is
>>> > > >>>>>>>>> > currently a lack of active contributors and committers
>>> who
>>> > are
>>> > > >>>>>>>>> able to help
>>> > > >>>>>>>>> > with the maintenance of the project.
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > In order to improve the situation, we need to solve the
>>> lack
>>> > of
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > committers and the lack of contributors.
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > On the lack of committers:
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > 1. Ideally, there are some of the current Flink
>>> committers
>>> > who
>>> > > >>>>>>>>> have
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > the bandwidth and can help with reviewing PRs and merging
>>> > them.
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > 2. If that's not an option, it could be a consideration
>>> that
>>> > > >>>>>>>>> current
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > committers only approve and review PRs, that are
>>> approved by
>>> > > >>>>>>>>> those who are
>>> > > >>>>>>>>> > willing to contribute to Statefun and if the CI passes
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > On the lack of contributors:
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > 3. Next to having this discussion on the Dev and User
>>> mailing
>>> > > >>>>>>>>> list, we
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > can also create a blog with a call for new contributors
>>> on
>>> > the
>>> > > >>>>>>>>> Flink
>>> > > >>>>>>>>> > project website, send out some tweets on the Flink /
>>> Statefun
>>> > > >>>>>>>>> twitter
>>> > > >>>>>>>>> > accounts, post messages on Slack etc. In that message, we
>>> > would
>>> > > >>>>>>>>> inform how
>>> > > >>>>>>>>> > those that are interested in contributing can start and
>>> where
>>> > > >>>>>>>>> they could
>>> > > >>>>>>>>> > reach out for more information.
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > There's also option 4. where a group of interested people
>>> > would
>>> > > >>>>>>>>> split
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > Statefun from the Flink project and make it a separate
>>> top
>>> > > level
>>> > > >>>>>>>>> project
>>> > > >>>>>>>>> > under the Apache Flink umbrella (similar as recently has
>>> > > >>>>>>>>> happened with
>>> > > >>>>>>>>> > Flink Table Store, which has become Apache Paimon).
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > If we see no improvements in the coming period, we should
>>> > > >>>>>>>>> consider
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > sunsetting Statefun and communicate that clearly to the
>>> > users.
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > I'm looking forward to your thoughts.
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > Best regards,
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > Martijn
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > [1]
>>> > > >>>>>>>>>
>>> https://nightlies.apache.org/flink/flink-statefun-docs-master/
>>> > <
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> >
>>> > https://nightlies.apache.org/flink/flink-statefun-docs-master/
>>> > > >
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > [2] https://github.com/apache/flink-statefun <
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> > https://github.com/apache/flink-statefun>
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>> >
>>> > > >>>>>>>>>
>>> > > >>>>>>>>
>>> > >
>>> >
>>>
>>

Re: [DISCUSS] Status of Statefun Project

["Tzu-Li (Gordon) Tai" <tz...@apache.org>]

Hi Galen,

The original intent of having a separate repo for the playground repo, was
that StateFun users can just go to that and start running simple examples
without any other distractions from the core code. I personally don't have
a strong preference here and can understand how it would make the workflow
more streamlined, but just FYI on the reasoning why are separate in the
first place.

re: paths for locating StateFun artifacts.
Can this be solved by simply passing in the path to the artifacts? As well
as the image tag for the locally build base StateFun image. They could
probably be environment variables.

Cheers,
Gordon

On Fri, Aug 18, 2023 at 12:13 PM Galen Warren via user <
user@flink.apache.org> wrote:

> Yes, exactly! And in addition to the base Statefun jars and the jar for
> the Java SDK, it does an equivalent copy/register operation for each of the
> other SDK libraries (Go, Python, Javascript) so that those libraries are
> also available when building the playground examples.
>
> One more question: In order to copy the various build artifacts into the
> Docker containers, those artifacts need to be part of the Docker context.
> With the playground being a separate project, that's slightly tricky to do,
> as there is no guarantee (other than convention) about the relative paths
> of *flink-statefun* and* flink-statefun-playground *in someone's local
> filesystem. The way I've set this up locally is to copy the playground into
> the* flink-statefun* project -- i.e. to *flink-statefun*/playground --
> such that I can set the Docker context to the root folder of
> *flink-statefun* and then have access to any local code and/or build
> artifacts.
>
> I'm wondering if there might be any appetite for making that move
> permanent, i.e. moving the playground to *flink-statefun*/playground and
> deprecating the standalone playground project. In addition to making the
> problem of building with unreleased artifacts a bit simpler to solve, it
> would also simplify the process of releasing a new Statefun version, since
> the entire process could be handled with a single PR and associated
> build/deploy tasks. In other words, a single PR could both update and
> deploy the Statefun package and the playground code and images.
>
> As it stands, at least two PRs would be required for each Statefun version
> update -- one for *flink-statefun* and one for *flink-statefun-playground*
> .
>
> Anyway, just an idea. Maybe there's an important reason for these projects
> to remain separate. If we do want to keep the playground project where it
> is, I could solve the copying problem by requiring the two projects to be
> siblings in the file system and by pre-copying the local build artifacts
> into a location accessible by the existing Docker contexts. This would
> still leave us with the need to have two PRs and releases instead of one,
> though.
>
> Thanks for your help!
>
>
> On Fri, Aug 18, 2023 at 2:45 PM Tzu-Li (Gordon) Tai <tz...@apache.org>
> wrote:
>
>> Hi Galen,
>>
>> > locally built code is copied into the build containers
>> so that it can be accessed during the build.
>>
>> That's exactly what we had been doing for release testing, yes. Sorry I
>> missed that detail in my previous response.
>>
>> And yes, that sounds like a reasonable approach. If I understand you
>> correctly, the workflow would become this:
>>
>>    1. Build the StateFun repo locally to install the snapshot artifact
>>    jars + have a local base StateFun image.
>>    2. Run the playground in "local" mode, so that it uses the local base
>>    StateFun image + builds the playground code using copied artifact jars
>>    (instead of pulling from Maven).
>>
>> That looks good to me!
>>
>> Thanks,
>> Gordon
>>
>> On Fri, Aug 18, 2023 at 11:33 AM Galen Warren
>> <ga...@cvillewarrens.com.invalid> wrote:
>>
>> > Thanks.
>> >
>> > If you were to build a local image, as you suggest, how do you access
>> that
>> > image when building the playground images? All the compilation of
>> > playground code happens inside containers, so local images on the host
>> > aren't available in those containers. Unless I'm missing something?
>> >
>> > I've slightly reworked things such that the playground images can be
>> run in
>> > one of two modes -- the default mode, which works like before, and a
>> > "local" mode where locally built code is copied into the build
>> containers
>> > so that it can be accessed during the build. It works fine, you just
>> have
>> > to define a couple of environment variables when running docker-compose
>> to
>> > specify default vs. local mode and what versions of Flink and Statefun
>> > should be referenced, and then you can build a run the local examples
>> > without any additional steps. Does that sound like a reasonable
>> approach?
>> >
>> >
>> > On Fri, Aug 18, 2023 at 2:17 PM Tzu-Li (Gordon) Tai <
>> tzulitai@apache.org>
>> > wrote:
>> >
>> > > Hi Galen,
>> > >
>> > > > Gordon, is there a trick to running the sample code in
>> > > flink-statefun-playground against yet-unreleased code that I'm
>> missing?
>> > >
>> > > You'd have to locally build an image from the release branch, with a
>> > > temporary image version tag. Then, in the flink-statefun-playground,
>> > change
>> > > the image versions in the docker-compose files to use that locally
>> built
>> > > image. IIRC, that's what we have been doing in the past. Admittedly,
>> it's
>> > > pretty manual - I don't think the CI manages this workflow.
>> > >
>> > > Thanks,
>> > > Gordon
>> > >
>> > > On Mon, Aug 14, 2023 at 10:42 AM Galen Warren <
>> galen@cvillewarrens.com>
>> > > wrote:
>> > >
>> > > > I created a pull request for this: [FLINK-31619] Upgrade Stateful
>> > > > Functions to Flink 1.16.1 by galenwarren · Pull Request #331 ·
>> > > > apache/flink-statefun (github.com)
>> > > > <https://github.com/apache/flink-statefun/pull/331>.
>> > > >
>> > > > JIRA is here: [FLINK-31619] Upgrade Stateful Functions to Flink
>> 1.16.1
>> > -
>> > > > ASF JIRA (apache.org)
>> > > > <https://issues.apache.org/jira/browse/FLINK-31619?filter=-1>.
>> > > >
>> > > > Statefun references 1.16.2, despite the title -- that version has
>> come
>> > > out
>> > > > since the issue was created.
>> > > >
>> > > > I figured out how to run all the playground tests locally, but it
>> took
>> > a
>> > > > bit of reworking of the playground setup with respect to Docker;
>> > > > specifically, the Docker contexts used to build the example
>> functions
>> > > > needed to be broadened (i.e. moved up the tree) so that, if needed,
>> > local
>> > > > artifacts/code can be accessed from within the containers at build
>> > time.
>> > > > Then I made the Docker compose.yml configurable through environment
>> > > > variables to allow for them to run in either the original manner --
>> > i.e.
>> > > > pulling artifacts from public repos -- or in a "local" mode, where
>> > > > artifacts are pulled from local builds.
>> > > >
>> > > > This process is a cleaner if the playground is a subfolder of the
>> > > > flink-statefun project rather than be its own repository
>> > > > (flink-statefun-playground), because then all the relative paths
>> > between
>> > > > the playground files and the build artifacts are fixed. So, I'd
>> like to
>> > > > propose to move the playground files, modified as described above,
>> to
>> > > > flink-statefun/playground and retire flink-statefun-playground. I
>> can
>> > > > submit separate PR s those changes if everyone is on board.
>> > > >
>> > > > Also, should I plan to do the same upgrade to handle Flink 1.17.x?
>> It
>> > > > should be easy to do, especially while the 1.16.x upgrade is fresh
>> on
>> > my
>> > > > mind.
>> > > >
>> > > > Thanks.
>> > > >
>> > > >
>> > > > On Fri, Aug 11, 2023 at 6:40 PM Galen Warren <
>> galen@cvillewarrens.com>
>> > > > wrote:
>> > > >
>> > > >> I'm done with the code to make Statefun compatible with Flink 1.16,
>> > and
>> > > >> all the tests (including e2e succeed). The required changes were
>> > pretty
>> > > >> minimal.
>> > > >>
>> > > >> I'm running into a bit of a chicken/egg problem executing the
>> tests in
>> > > >> flink-statefun-playground
>> > > >> <https://github.com/apache/flink-statefun-playground>, though.
>> That
>> > > >> project seems to assume that all the various Statefun artifacts are
>> > > built
>> > > >> and deployed to the various public repositories already. I've
>> looked
>> > > into
>> > > >> trying to redirect references to local artifacts; however, that's
>> also
>> > > >> tricky since all the building occurs in Docker containers.
>> > > >>
>> > > >> Gordon, is there a trick to running the sample code in
>> > > >> flink-statefun-playground against yet-unreleased code that I'm
>> > missing?
>> > > >>
>> > > >> Thanks.
>> > > >>
>> > > >> On Sat, Jun 24, 2023 at 12:40 PM Galen Warren <
>> > galen@cvillewarrens.com>
>> > > >> wrote:
>> > > >>
>> > > >>> Great -- thanks!
>> > > >>>
>> > > >>> I'm going to be out of town for about a week but I'll take a look
>> at
>> > > >>> this when I'm back.
>> > > >>>
>> > > >>> On Tue, Jun 20, 2023 at 8:46 AM Martijn Visser <
>> mvisser@confluent.io
>> > >
>> > > >>> wrote:
>> > > >>>
>> > > >>>> Hi Galen,
>> > > >>>>
>> > > >>>> Yes, I'll be more than happy to help with Statefun releases.
>> > > >>>>
>> > > >>>> Best regards,
>> > > >>>>
>> > > >>>> Martijn
>> > > >>>>
>> > > >>>> On Tue, Jun 20, 2023 at 2:21 PM Galen Warren <
>> > galen@cvillewarrens.com
>> > > >
>> > > >>>> wrote:
>> > > >>>>
>> > > >>>>> Thanks.
>> > > >>>>>
>> > > >>>>> Martijn, to answer your question, I'd need to do a small amount
>> of
>> > > >>>>> work to get a PR ready, but not much. Happy to do it if we're
>> > > deciding to
>> > > >>>>> restart Statefun releases -- are we?
>> > > >>>>>
>> > > >>>>> -- Galen
>> > > >>>>>
>> > > >>>>> On Sat, Jun 17, 2023 at 9:47 AM Tzu-Li (Gordon) Tai <
>> > > >>>>> tzulitai@apache.org> wrote:
>> > > >>>>>
>> > > >>>>>> > Perhaps he could weigh in on whether the combination of
>> > automated
>> > > >>>>>> tests plus those smoke tests should be sufficient for testing
>> with
>> > > new
>> > > >>>>>> Flink versions
>> > > >>>>>>
>> > > >>>>>> What we usually did at the bare minimum for new StateFun
>> releases
>> > > was
>> > > >>>>>> the following:
>> > > >>>>>>
>> > > >>>>>>    1. Build tests (including the smoke tests in the e2e module,
>> > > >>>>>>    which covers important tests like exactly-once verification)
>> > > >>>>>>    2. Updating the flink-statefun-playground repo and manually
>> > > >>>>>>    running all language examples there.
>> > > >>>>>>
>> > > >>>>>> If upgrading Flink versions was the only change in the release,
>> > I'd
>> > > >>>>>> probably say that this is sufficient.
>> > > >>>>>>
>> > > >>>>>> Best,
>> > > >>>>>> Gordon
>> > > >>>>>>
>> > > >>>>>> On Thu, Jun 15, 2023 at 5:25 AM Martijn Visser <
>> > > >>>>>> martijnvisser@apache.org> wrote:
>> > > >>>>>>
>> > > >>>>>>> Let me know if you have a PR for a Flink update :)
>> > > >>>>>>>
>> > > >>>>>>> On Thu, Jun 8, 2023 at 5:52 PM Galen Warren via user <
>> > > >>>>>>> user@flink.apache.org> wrote:
>> > > >>>>>>>
>> > > >>>>>>>> Thanks Martijn.
>> > > >>>>>>>>
>> > > >>>>>>>> Personally, I'm already using a local fork of Statefun that
>> is
>> > > >>>>>>>> compatible with Flink 1.16.x, so I wouldn't have any need
>> for a
>> > > released
>> > > >>>>>>>> version compatible with 1.15.x. I'd be happy to do the PRs to
>> > > modify
>> > > >>>>>>>> Statefun to work with new versions of Flink as they come
>> along.
>> > > >>>>>>>>
>> > > >>>>>>>> As for testing, Statefun does have unit tests and Gordon also
>> > sent
>> > > >>>>>>>> me instructions a while back for how to do some additional
>> smoke
>> > > tests
>> > > >>>>>>>> which are pretty straightforward. Perhaps he could weigh in
>> on
>> > > whether the
>> > > >>>>>>>> combination of automated tests plus those smoke tests should
>> be
>> > > sufficient
>> > > >>>>>>>> for testing with new Flink versions (I believe the answer is
>> > yes).
>> > > >>>>>>>>
>> > > >>>>>>>> -- Galen
>> > > >>>>>>>>
>> > > >>>>>>>>
>> > > >>>>>>>>
>> > > >>>>>>>> On Thu, Jun 8, 2023 at 8:01 AM Martijn Visser <
>> > > >>>>>>>> martijnvisser@apache.org> wrote:
>> > > >>>>>>>>
>> > > >>>>>>>>> Hi all,
>> > > >>>>>>>>>
>> > > >>>>>>>>> Apologies for the late reply.
>> > > >>>>>>>>>
>> > > >>>>>>>>> I'm willing to help out with merging requests in Statefun to
>> > keep
>> > > >>>>>>>>> them
>> > > >>>>>>>>> compatible with new Flink releases and create new releases.
>> I
>> > do
>> > > >>>>>>>>> think that
>> > > >>>>>>>>> validation of the functionality of these releases depends a
>> lot
>> > > on
>> > > >>>>>>>>> those
>> > > >>>>>>>>> who do these compatibility updates, with PMC members helping
>> > out
>> > > >>>>>>>>> with the
>> > > >>>>>>>>> formal process.
>> > > >>>>>>>>>
>> > > >>>>>>>>> > Why can't the Apache Software Foundation allow community
>> > > members
>> > > >>>>>>>>> to bring
>> > > >>>>>>>>> it up to date?
>> > > >>>>>>>>>
>> > > >>>>>>>>> There's nothing preventing anyone from reviewing any of the
>> > > >>>>>>>>> current PRs or
>> > > >>>>>>>>> opening new ones. However, none of them are approved [1], so
>> > > >>>>>>>>> there's also
>> > > >>>>>>>>> nothing to merge.
>> > > >>>>>>>>>
>> > > >>>>>>>>> > I believe that there are people and companies on this
>> mailing
>> > > >>>>>>>>> list
>> > > >>>>>>>>> interested in supporting Apache Flink Stateful Functions.
>> > > >>>>>>>>>
>> > > >>>>>>>>> If so, then now is the time to show.
>> > > >>>>>>>>>
>> > > >>>>>>>>> Would there be a preference to create a release with Galen's
>> > > merged
>> > > >>>>>>>>> compatibility update to Flink 1.15.2, or do we want to skip
>> > that
>> > > >>>>>>>>> and go
>> > > >>>>>>>>> straight to a newer version?
>> > > >>>>>>>>>
>> > > >>>>>>>>> Best regards,
>> > > >>>>>>>>>
>> > > >>>>>>>>> Martijn
>> > > >>>>>>>>>
>> > > >>>>>>>>> [1]
>> > > >>>>>>>>>
>> > > >>>>>>>>>
>> > >
>> >
>> https://github.com/apache/flink-statefun/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved
>> > > >>>>>>>>>
>> > > >>>>>>>>> On Tue, Jun 6, 2023 at 3:55 PM Marco Villalobos <
>> > > >>>>>>>>> mvillalobos@kineteque.com>
>> > > >>>>>>>>> wrote:
>> > > >>>>>>>>>
>> > > >>>>>>>>> > Why can't the Apache Software Foundation allow community
>> > > members
>> > > >>>>>>>>> to bring
>> > > >>>>>>>>> > it up to date?
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > What's the process for that?
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > I believe that there are people and companies on this
>> mailing
>> > > >>>>>>>>> list
>> > > >>>>>>>>> > interested in supporting Apache Flink Stateful Functions.
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > You already had two people on this thread express
>> interest.
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > At the very least, we could keep the library versions up
>> to
>> > > date.
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > There are only a small list of new features that might be
>> > > >>>>>>>>> worthwhile:
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > 1. event time processing
>> > > >>>>>>>>> > 2. state rest api
>> > > >>>>>>>>> >
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <
>> > > chesnay@apache.org>
>> > > >>>>>>>>> wrote:
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > If you were to fork it *and want to redistribute it* then
>> the
>> > > >>>>>>>>> short
>> > > >>>>>>>>> > version is that
>> > > >>>>>>>>> >
>> > > >>>>>>>>> >    1. you have to adhere to the Apache licensing
>> requirements
>> > > >>>>>>>>> >    2. you have to make it clear that your fork does not
>> > belong
>> > > >>>>>>>>> to the
>> > > >>>>>>>>> >    Apache Flink project. (Trademarks and all that)
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > Neither should be significant hurdles (there should also
>> be
>> > > >>>>>>>>> plenty of
>> > > >>>>>>>>> > online resources regarding 1), and if you do this then you
>> > can
>> > > >>>>>>>>> freely share
>> > > >>>>>>>>> > your fork with others.
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > I've also pinged Martijn to take a look at this thread.
>> > > >>>>>>>>> > To my knowledge the project hasn't decided anything yet.
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > On 27/05/2023 04:05, Galen Warren wrote:
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > Ok, I get it. No interest.
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > If this project is being abandoned, I guess I'll work
>> with my
>> > > >>>>>>>>> own fork. Is
>> > > >>>>>>>>> > there anything I should consider here? Can I share it with
>> > > other
>> > > >>>>>>>>> people who
>> > > >>>>>>>>> > use this project?
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > On Tue, May 16, 2023 at 10:50 AM Galen Warren <
>> > > >>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
>> > > >>>>>>>>> > wrote:
>> > > >>>>>>>>> >
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > Hi Martijn, since you opened this discussion thread, I'm
>> > > curious
>> > > >>>>>>>>> what your
>> > > >>>>>>>>> > thoughts are in light of the responses? Thanks.
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <
>> > > >>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
>> > > >>>>>>>>> > wrote:
>> > > >>>>>>>>> >
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > I use Apache Flink for stream processing, and StateFun as
>> a
>> > > >>>>>>>>> hand-off
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > point for the rest of the application.
>> > > >>>>>>>>> > It serves well as a bridge between a Flink Streaming job
>> and
>> > > >>>>>>>>> > micro-services.
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > This is essentially how I use it as well, and I would
>> also be
>> > > >>>>>>>>> sad to see
>> > > >>>>>>>>> > it sunsetted. It works well; I don't know that there is a
>> lot
>> > > of
>> > > >>>>>>>>> new
>> > > >>>>>>>>> > development required, but if there are no new Statefun
>> > > releases,
>> > > >>>>>>>>> then
>> > > >>>>>>>>> > Statefun can only be used with older Flink versions.
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
>> > > >>>>>>>>> mvillalobos@kineteque.com> wrote:
>> > > >>>>>>>>> >
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > I am currently using Stateful Functions in my application.
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > I use Apache Flink for stream processing, and StateFun as
>> a
>> > > >>>>>>>>> hand-off
>> > > >>>>>>>>> > point for the rest of the application.
>> > > >>>>>>>>> > It serves well as a bridge between a Flink Streaming job
>> and
>> > > >>>>>>>>> > micro-services.
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > I would be disappointed if StateFun was sunsetted.  Its a
>> > good
>> > > >>>>>>>>> idea.
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > If there is anything I can do to help, as a contributor
>> > > perhaps,
>> > > >>>>>>>>> please
>> > > >>>>>>>>> > let me know.
>> > > >>>>>>>>> >
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <
>> > > >>>>>>>>> martijnvisser@apache.org> <ma...@apache.org>
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > wrote:
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > Hi everyone,
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > I want to open a discussion on the status of the Statefun
>> > > >>>>>>>>> Project [1]
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > in Apache Flink. As you might have noticed, there hasn't
>> been
>> > > >>>>>>>>> much
>> > > >>>>>>>>> > development over the past months in the Statefun
>> repository
>> > > [2].
>> > > >>>>>>>>> There is
>> > > >>>>>>>>> > currently a lack of active contributors and committers who
>> > are
>> > > >>>>>>>>> able to help
>> > > >>>>>>>>> > with the maintenance of the project.
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > In order to improve the situation, we need to solve the
>> lack
>> > of
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > committers and the lack of contributors.
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > On the lack of committers:
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > 1. Ideally, there are some of the current Flink committers
>> > who
>> > > >>>>>>>>> have
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > the bandwidth and can help with reviewing PRs and merging
>> > them.
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > 2. If that's not an option, it could be a consideration
>> that
>> > > >>>>>>>>> current
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > committers only approve and review PRs, that are approved
>> by
>> > > >>>>>>>>> those who are
>> > > >>>>>>>>> > willing to contribute to Statefun and if the CI passes
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > On the lack of contributors:
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > 3. Next to having this discussion on the Dev and User
>> mailing
>> > > >>>>>>>>> list, we
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > can also create a blog with a call for new contributors on
>> > the
>> > > >>>>>>>>> Flink
>> > > >>>>>>>>> > project website, send out some tweets on the Flink /
>> Statefun
>> > > >>>>>>>>> twitter
>> > > >>>>>>>>> > accounts, post messages on Slack etc. In that message, we
>> > would
>> > > >>>>>>>>> inform how
>> > > >>>>>>>>> > those that are interested in contributing can start and
>> where
>> > > >>>>>>>>> they could
>> > > >>>>>>>>> > reach out for more information.
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > There's also option 4. where a group of interested people
>> > would
>> > > >>>>>>>>> split
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > Statefun from the Flink project and make it a separate top
>> > > level
>> > > >>>>>>>>> project
>> > > >>>>>>>>> > under the Apache Flink umbrella (similar as recently has
>> > > >>>>>>>>> happened with
>> > > >>>>>>>>> > Flink Table Store, which has become Apache Paimon).
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > If we see no improvements in the coming period, we should
>> > > >>>>>>>>> consider
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > sunsetting Statefun and communicate that clearly to the
>> > users.
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > I'm looking forward to your thoughts.
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > Best regards,
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > Martijn
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > [1]
>> > > >>>>>>>>>
>> https://nightlies.apache.org/flink/flink-statefun-docs-master/
>> > <
>> > > >>>>>>>>> >
>> > > >>>>>>>>> >
>> > https://nightlies.apache.org/flink/flink-statefun-docs-master/
>> > > >
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > [2] https://github.com/apache/flink-statefun <
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > https://github.com/apache/flink-statefun>
>> > > >>>>>>>>> >
>> > > >>>>>>>>> >
>> > > >>>>>>>>> >
>> > > >>>>>>>>> >
>> > > >>>>>>>>> >
>> > > >>>>>>>>>
>> > > >>>>>>>>
>> > >
>> >
>>
>

Re: [DISCUSS] Status of Statefun Project

["Tzu-Li (Gordon) Tai" <tz...@apache.org>]

Hi Galen,

The original intent of having a separate repo for the playground repo, was
that StateFun users can just go to that and start running simple examples
without any other distractions from the core code. I personally don't have
a strong preference here and can understand how it would make the workflow
more streamlined, but just FYI on the reasoning why are separate in the
first place.

re: paths for locating StateFun artifacts.
Can this be solved by simply passing in the path to the artifacts? As well
as the image tag for the locally build base StateFun image. They could
probably be environment variables.

Cheers,
Gordon

On Fri, Aug 18, 2023 at 12:13 PM Galen Warren via user <
user@flink.apache.org> wrote:

> Yes, exactly! And in addition to the base Statefun jars and the jar for
> the Java SDK, it does an equivalent copy/register operation for each of the
> other SDK libraries (Go, Python, Javascript) so that those libraries are
> also available when building the playground examples.
>
> One more question: In order to copy the various build artifacts into the
> Docker containers, those artifacts need to be part of the Docker context.
> With the playground being a separate project, that's slightly tricky to do,
> as there is no guarantee (other than convention) about the relative paths
> of *flink-statefun* and* flink-statefun-playground *in someone's local
> filesystem. The way I've set this up locally is to copy the playground into
> the* flink-statefun* project -- i.e. to *flink-statefun*/playground --
> such that I can set the Docker context to the root folder of
> *flink-statefun* and then have access to any local code and/or build
> artifacts.
>
> I'm wondering if there might be any appetite for making that move
> permanent, i.e. moving the playground to *flink-statefun*/playground and
> deprecating the standalone playground project. In addition to making the
> problem of building with unreleased artifacts a bit simpler to solve, it
> would also simplify the process of releasing a new Statefun version, since
> the entire process could be handled with a single PR and associated
> build/deploy tasks. In other words, a single PR could both update and
> deploy the Statefun package and the playground code and images.
>
> As it stands, at least two PRs would be required for each Statefun version
> update -- one for *flink-statefun* and one for *flink-statefun-playground*
> .
>
> Anyway, just an idea. Maybe there's an important reason for these projects
> to remain separate. If we do want to keep the playground project where it
> is, I could solve the copying problem by requiring the two projects to be
> siblings in the file system and by pre-copying the local build artifacts
> into a location accessible by the existing Docker contexts. This would
> still leave us with the need to have two PRs and releases instead of one,
> though.
>
> Thanks for your help!
>
>
> On Fri, Aug 18, 2023 at 2:45 PM Tzu-Li (Gordon) Tai <tz...@apache.org>
> wrote:
>
>> Hi Galen,
>>
>> > locally built code is copied into the build containers
>> so that it can be accessed during the build.
>>
>> That's exactly what we had been doing for release testing, yes. Sorry I
>> missed that detail in my previous response.
>>
>> And yes, that sounds like a reasonable approach. If I understand you
>> correctly, the workflow would become this:
>>
>>    1. Build the StateFun repo locally to install the snapshot artifact
>>    jars + have a local base StateFun image.
>>    2. Run the playground in "local" mode, so that it uses the local base
>>    StateFun image + builds the playground code using copied artifact jars
>>    (instead of pulling from Maven).
>>
>> That looks good to me!
>>
>> Thanks,
>> Gordon
>>
>> On Fri, Aug 18, 2023 at 11:33 AM Galen Warren
>> <ga...@cvillewarrens.com.invalid> wrote:
>>
>> > Thanks.
>> >
>> > If you were to build a local image, as you suggest, how do you access
>> that
>> > image when building the playground images? All the compilation of
>> > playground code happens inside containers, so local images on the host
>> > aren't available in those containers. Unless I'm missing something?
>> >
>> > I've slightly reworked things such that the playground images can be
>> run in
>> > one of two modes -- the default mode, which works like before, and a
>> > "local" mode where locally built code is copied into the build
>> containers
>> > so that it can be accessed during the build. It works fine, you just
>> have
>> > to define a couple of environment variables when running docker-compose
>> to
>> > specify default vs. local mode and what versions of Flink and Statefun
>> > should be referenced, and then you can build a run the local examples
>> > without any additional steps. Does that sound like a reasonable
>> approach?
>> >
>> >
>> > On Fri, Aug 18, 2023 at 2:17 PM Tzu-Li (Gordon) Tai <
>> tzulitai@apache.org>
>> > wrote:
>> >
>> > > Hi Galen,
>> > >
>> > > > Gordon, is there a trick to running the sample code in
>> > > flink-statefun-playground against yet-unreleased code that I'm
>> missing?
>> > >
>> > > You'd have to locally build an image from the release branch, with a
>> > > temporary image version tag. Then, in the flink-statefun-playground,
>> > change
>> > > the image versions in the docker-compose files to use that locally
>> built
>> > > image. IIRC, that's what we have been doing in the past. Admittedly,
>> it's
>> > > pretty manual - I don't think the CI manages this workflow.
>> > >
>> > > Thanks,
>> > > Gordon
>> > >
>> > > On Mon, Aug 14, 2023 at 10:42 AM Galen Warren <
>> galen@cvillewarrens.com>
>> > > wrote:
>> > >
>> > > > I created a pull request for this: [FLINK-31619] Upgrade Stateful
>> > > > Functions to Flink 1.16.1 by galenwarren · Pull Request #331 ·
>> > > > apache/flink-statefun (github.com)
>> > > > <https://github.com/apache/flink-statefun/pull/331>.
>> > > >
>> > > > JIRA is here: [FLINK-31619] Upgrade Stateful Functions to Flink
>> 1.16.1
>> > -
>> > > > ASF JIRA (apache.org)
>> > > > <https://issues.apache.org/jira/browse/FLINK-31619?filter=-1>.
>> > > >
>> > > > Statefun references 1.16.2, despite the title -- that version has
>> come
>> > > out
>> > > > since the issue was created.
>> > > >
>> > > > I figured out how to run all the playground tests locally, but it
>> took
>> > a
>> > > > bit of reworking of the playground setup with respect to Docker;
>> > > > specifically, the Docker contexts used to build the example
>> functions
>> > > > needed to be broadened (i.e. moved up the tree) so that, if needed,
>> > local
>> > > > artifacts/code can be accessed from within the containers at build
>> > time.
>> > > > Then I made the Docker compose.yml configurable through environment
>> > > > variables to allow for them to run in either the original manner --
>> > i.e.
>> > > > pulling artifacts from public repos -- or in a "local" mode, where
>> > > > artifacts are pulled from local builds.
>> > > >
>> > > > This process is a cleaner if the playground is a subfolder of the
>> > > > flink-statefun project rather than be its own repository
>> > > > (flink-statefun-playground), because then all the relative paths
>> > between
>> > > > the playground files and the build artifacts are fixed. So, I'd
>> like to
>> > > > propose to move the playground files, modified as described above,
>> to
>> > > > flink-statefun/playground and retire flink-statefun-playground. I
>> can
>> > > > submit separate PR s those changes if everyone is on board.
>> > > >
>> > > > Also, should I plan to do the same upgrade to handle Flink 1.17.x?
>> It
>> > > > should be easy to do, especially while the 1.16.x upgrade is fresh
>> on
>> > my
>> > > > mind.
>> > > >
>> > > > Thanks.
>> > > >
>> > > >
>> > > > On Fri, Aug 11, 2023 at 6:40 PM Galen Warren <
>> galen@cvillewarrens.com>
>> > > > wrote:
>> > > >
>> > > >> I'm done with the code to make Statefun compatible with Flink 1.16,
>> > and
>> > > >> all the tests (including e2e succeed). The required changes were
>> > pretty
>> > > >> minimal.
>> > > >>
>> > > >> I'm running into a bit of a chicken/egg problem executing the
>> tests in
>> > > >> flink-statefun-playground
>> > > >> <https://github.com/apache/flink-statefun-playground>, though.
>> That
>> > > >> project seems to assume that all the various Statefun artifacts are
>> > > built
>> > > >> and deployed to the various public repositories already. I've
>> looked
>> > > into
>> > > >> trying to redirect references to local artifacts; however, that's
>> also
>> > > >> tricky since all the building occurs in Docker containers.
>> > > >>
>> > > >> Gordon, is there a trick to running the sample code in
>> > > >> flink-statefun-playground against yet-unreleased code that I'm
>> > missing?
>> > > >>
>> > > >> Thanks.
>> > > >>
>> > > >> On Sat, Jun 24, 2023 at 12:40 PM Galen Warren <
>> > galen@cvillewarrens.com>
>> > > >> wrote:
>> > > >>
>> > > >>> Great -- thanks!
>> > > >>>
>> > > >>> I'm going to be out of town for about a week but I'll take a look
>> at
>> > > >>> this when I'm back.
>> > > >>>
>> > > >>> On Tue, Jun 20, 2023 at 8:46 AM Martijn Visser <
>> mvisser@confluent.io
>> > >
>> > > >>> wrote:
>> > > >>>
>> > > >>>> Hi Galen,
>> > > >>>>
>> > > >>>> Yes, I'll be more than happy to help with Statefun releases.
>> > > >>>>
>> > > >>>> Best regards,
>> > > >>>>
>> > > >>>> Martijn
>> > > >>>>
>> > > >>>> On Tue, Jun 20, 2023 at 2:21 PM Galen Warren <
>> > galen@cvillewarrens.com
>> > > >
>> > > >>>> wrote:
>> > > >>>>
>> > > >>>>> Thanks.
>> > > >>>>>
>> > > >>>>> Martijn, to answer your question, I'd need to do a small amount
>> of
>> > > >>>>> work to get a PR ready, but not much. Happy to do it if we're
>> > > deciding to
>> > > >>>>> restart Statefun releases -- are we?
>> > > >>>>>
>> > > >>>>> -- Galen
>> > > >>>>>
>> > > >>>>> On Sat, Jun 17, 2023 at 9:47 AM Tzu-Li (Gordon) Tai <
>> > > >>>>> tzulitai@apache.org> wrote:
>> > > >>>>>
>> > > >>>>>> > Perhaps he could weigh in on whether the combination of
>> > automated
>> > > >>>>>> tests plus those smoke tests should be sufficient for testing
>> with
>> > > new
>> > > >>>>>> Flink versions
>> > > >>>>>>
>> > > >>>>>> What we usually did at the bare minimum for new StateFun
>> releases
>> > > was
>> > > >>>>>> the following:
>> > > >>>>>>
>> > > >>>>>>    1. Build tests (including the smoke tests in the e2e module,
>> > > >>>>>>    which covers important tests like exactly-once verification)
>> > > >>>>>>    2. Updating the flink-statefun-playground repo and manually
>> > > >>>>>>    running all language examples there.
>> > > >>>>>>
>> > > >>>>>> If upgrading Flink versions was the only change in the release,
>> > I'd
>> > > >>>>>> probably say that this is sufficient.
>> > > >>>>>>
>> > > >>>>>> Best,
>> > > >>>>>> Gordon
>> > > >>>>>>
>> > > >>>>>> On Thu, Jun 15, 2023 at 5:25 AM Martijn Visser <
>> > > >>>>>> martijnvisser@apache.org> wrote:
>> > > >>>>>>
>> > > >>>>>>> Let me know if you have a PR for a Flink update :)
>> > > >>>>>>>
>> > > >>>>>>> On Thu, Jun 8, 2023 at 5:52 PM Galen Warren via user <
>> > > >>>>>>> user@flink.apache.org> wrote:
>> > > >>>>>>>
>> > > >>>>>>>> Thanks Martijn.
>> > > >>>>>>>>
>> > > >>>>>>>> Personally, I'm already using a local fork of Statefun that
>> is
>> > > >>>>>>>> compatible with Flink 1.16.x, so I wouldn't have any need
>> for a
>> > > released
>> > > >>>>>>>> version compatible with 1.15.x. I'd be happy to do the PRs to
>> > > modify
>> > > >>>>>>>> Statefun to work with new versions of Flink as they come
>> along.
>> > > >>>>>>>>
>> > > >>>>>>>> As for testing, Statefun does have unit tests and Gordon also
>> > sent
>> > > >>>>>>>> me instructions a while back for how to do some additional
>> smoke
>> > > tests
>> > > >>>>>>>> which are pretty straightforward. Perhaps he could weigh in
>> on
>> > > whether the
>> > > >>>>>>>> combination of automated tests plus those smoke tests should
>> be
>> > > sufficient
>> > > >>>>>>>> for testing with new Flink versions (I believe the answer is
>> > yes).
>> > > >>>>>>>>
>> > > >>>>>>>> -- Galen
>> > > >>>>>>>>
>> > > >>>>>>>>
>> > > >>>>>>>>
>> > > >>>>>>>> On Thu, Jun 8, 2023 at 8:01 AM Martijn Visser <
>> > > >>>>>>>> martijnvisser@apache.org> wrote:
>> > > >>>>>>>>
>> > > >>>>>>>>> Hi all,
>> > > >>>>>>>>>
>> > > >>>>>>>>> Apologies for the late reply.
>> > > >>>>>>>>>
>> > > >>>>>>>>> I'm willing to help out with merging requests in Statefun to
>> > keep
>> > > >>>>>>>>> them
>> > > >>>>>>>>> compatible with new Flink releases and create new releases.
>> I
>> > do
>> > > >>>>>>>>> think that
>> > > >>>>>>>>> validation of the functionality of these releases depends a
>> lot
>> > > on
>> > > >>>>>>>>> those
>> > > >>>>>>>>> who do these compatibility updates, with PMC members helping
>> > out
>> > > >>>>>>>>> with the
>> > > >>>>>>>>> formal process.
>> > > >>>>>>>>>
>> > > >>>>>>>>> > Why can't the Apache Software Foundation allow community
>> > > members
>> > > >>>>>>>>> to bring
>> > > >>>>>>>>> it up to date?
>> > > >>>>>>>>>
>> > > >>>>>>>>> There's nothing preventing anyone from reviewing any of the
>> > > >>>>>>>>> current PRs or
>> > > >>>>>>>>> opening new ones. However, none of them are approved [1], so
>> > > >>>>>>>>> there's also
>> > > >>>>>>>>> nothing to merge.
>> > > >>>>>>>>>
>> > > >>>>>>>>> > I believe that there are people and companies on this
>> mailing
>> > > >>>>>>>>> list
>> > > >>>>>>>>> interested in supporting Apache Flink Stateful Functions.
>> > > >>>>>>>>>
>> > > >>>>>>>>> If so, then now is the time to show.
>> > > >>>>>>>>>
>> > > >>>>>>>>> Would there be a preference to create a release with Galen's
>> > > merged
>> > > >>>>>>>>> compatibility update to Flink 1.15.2, or do we want to skip
>> > that
>> > > >>>>>>>>> and go
>> > > >>>>>>>>> straight to a newer version?
>> > > >>>>>>>>>
>> > > >>>>>>>>> Best regards,
>> > > >>>>>>>>>
>> > > >>>>>>>>> Martijn
>> > > >>>>>>>>>
>> > > >>>>>>>>> [1]
>> > > >>>>>>>>>
>> > > >>>>>>>>>
>> > >
>> >
>> https://github.com/apache/flink-statefun/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved
>> > > >>>>>>>>>
>> > > >>>>>>>>> On Tue, Jun 6, 2023 at 3:55 PM Marco Villalobos <
>> > > >>>>>>>>> mvillalobos@kineteque.com>
>> > > >>>>>>>>> wrote:
>> > > >>>>>>>>>
>> > > >>>>>>>>> > Why can't the Apache Software Foundation allow community
>> > > members
>> > > >>>>>>>>> to bring
>> > > >>>>>>>>> > it up to date?
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > What's the process for that?
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > I believe that there are people and companies on this
>> mailing
>> > > >>>>>>>>> list
>> > > >>>>>>>>> > interested in supporting Apache Flink Stateful Functions.
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > You already had two people on this thread express
>> interest.
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > At the very least, we could keep the library versions up
>> to
>> > > date.
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > There are only a small list of new features that might be
>> > > >>>>>>>>> worthwhile:
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > 1. event time processing
>> > > >>>>>>>>> > 2. state rest api
>> > > >>>>>>>>> >
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <
>> > > chesnay@apache.org>
>> > > >>>>>>>>> wrote:
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > If you were to fork it *and want to redistribute it* then
>> the
>> > > >>>>>>>>> short
>> > > >>>>>>>>> > version is that
>> > > >>>>>>>>> >
>> > > >>>>>>>>> >    1. you have to adhere to the Apache licensing
>> requirements
>> > > >>>>>>>>> >    2. you have to make it clear that your fork does not
>> > belong
>> > > >>>>>>>>> to the
>> > > >>>>>>>>> >    Apache Flink project. (Trademarks and all that)
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > Neither should be significant hurdles (there should also
>> be
>> > > >>>>>>>>> plenty of
>> > > >>>>>>>>> > online resources regarding 1), and if you do this then you
>> > can
>> > > >>>>>>>>> freely share
>> > > >>>>>>>>> > your fork with others.
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > I've also pinged Martijn to take a look at this thread.
>> > > >>>>>>>>> > To my knowledge the project hasn't decided anything yet.
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > On 27/05/2023 04:05, Galen Warren wrote:
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > Ok, I get it. No interest.
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > If this project is being abandoned, I guess I'll work
>> with my
>> > > >>>>>>>>> own fork. Is
>> > > >>>>>>>>> > there anything I should consider here? Can I share it with
>> > > other
>> > > >>>>>>>>> people who
>> > > >>>>>>>>> > use this project?
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > On Tue, May 16, 2023 at 10:50 AM Galen Warren <
>> > > >>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
>> > > >>>>>>>>> > wrote:
>> > > >>>>>>>>> >
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > Hi Martijn, since you opened this discussion thread, I'm
>> > > curious
>> > > >>>>>>>>> what your
>> > > >>>>>>>>> > thoughts are in light of the responses? Thanks.
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <
>> > > >>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
>> > > >>>>>>>>> > wrote:
>> > > >>>>>>>>> >
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > I use Apache Flink for stream processing, and StateFun as
>> a
>> > > >>>>>>>>> hand-off
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > point for the rest of the application.
>> > > >>>>>>>>> > It serves well as a bridge between a Flink Streaming job
>> and
>> > > >>>>>>>>> > micro-services.
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > This is essentially how I use it as well, and I would
>> also be
>> > > >>>>>>>>> sad to see
>> > > >>>>>>>>> > it sunsetted. It works well; I don't know that there is a
>> lot
>> > > of
>> > > >>>>>>>>> new
>> > > >>>>>>>>> > development required, but if there are no new Statefun
>> > > releases,
>> > > >>>>>>>>> then
>> > > >>>>>>>>> > Statefun can only be used with older Flink versions.
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
>> > > >>>>>>>>> mvillalobos@kineteque.com> wrote:
>> > > >>>>>>>>> >
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > I am currently using Stateful Functions in my application.
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > I use Apache Flink for stream processing, and StateFun as
>> a
>> > > >>>>>>>>> hand-off
>> > > >>>>>>>>> > point for the rest of the application.
>> > > >>>>>>>>> > It serves well as a bridge between a Flink Streaming job
>> and
>> > > >>>>>>>>> > micro-services.
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > I would be disappointed if StateFun was sunsetted.  Its a
>> > good
>> > > >>>>>>>>> idea.
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > If there is anything I can do to help, as a contributor
>> > > perhaps,
>> > > >>>>>>>>> please
>> > > >>>>>>>>> > let me know.
>> > > >>>>>>>>> >
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <
>> > > >>>>>>>>> martijnvisser@apache.org> <ma...@apache.org>
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > wrote:
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > Hi everyone,
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > I want to open a discussion on the status of the Statefun
>> > > >>>>>>>>> Project [1]
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > in Apache Flink. As you might have noticed, there hasn't
>> been
>> > > >>>>>>>>> much
>> > > >>>>>>>>> > development over the past months in the Statefun
>> repository
>> > > [2].
>> > > >>>>>>>>> There is
>> > > >>>>>>>>> > currently a lack of active contributors and committers who
>> > are
>> > > >>>>>>>>> able to help
>> > > >>>>>>>>> > with the maintenance of the project.
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > In order to improve the situation, we need to solve the
>> lack
>> > of
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > committers and the lack of contributors.
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > On the lack of committers:
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > 1. Ideally, there are some of the current Flink committers
>> > who
>> > > >>>>>>>>> have
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > the bandwidth and can help with reviewing PRs and merging
>> > them.
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > 2. If that's not an option, it could be a consideration
>> that
>> > > >>>>>>>>> current
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > committers only approve and review PRs, that are approved
>> by
>> > > >>>>>>>>> those who are
>> > > >>>>>>>>> > willing to contribute to Statefun and if the CI passes
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > On the lack of contributors:
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > 3. Next to having this discussion on the Dev and User
>> mailing
>> > > >>>>>>>>> list, we
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > can also create a blog with a call for new contributors on
>> > the
>> > > >>>>>>>>> Flink
>> > > >>>>>>>>> > project website, send out some tweets on the Flink /
>> Statefun
>> > > >>>>>>>>> twitter
>> > > >>>>>>>>> > accounts, post messages on Slack etc. In that message, we
>> > would
>> > > >>>>>>>>> inform how
>> > > >>>>>>>>> > those that are interested in contributing can start and
>> where
>> > > >>>>>>>>> they could
>> > > >>>>>>>>> > reach out for more information.
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > There's also option 4. where a group of interested people
>> > would
>> > > >>>>>>>>> split
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > Statefun from the Flink project and make it a separate top
>> > > level
>> > > >>>>>>>>> project
>> > > >>>>>>>>> > under the Apache Flink umbrella (similar as recently has
>> > > >>>>>>>>> happened with
>> > > >>>>>>>>> > Flink Table Store, which has become Apache Paimon).
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > If we see no improvements in the coming period, we should
>> > > >>>>>>>>> consider
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > sunsetting Statefun and communicate that clearly to the
>> > users.
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > I'm looking forward to your thoughts.
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > Best regards,
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > Martijn
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > [1]
>> > > >>>>>>>>>
>> https://nightlies.apache.org/flink/flink-statefun-docs-master/
>> > <
>> > > >>>>>>>>> >
>> > > >>>>>>>>> >
>> > https://nightlies.apache.org/flink/flink-statefun-docs-master/
>> > > >
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > [2] https://github.com/apache/flink-statefun <
>> > > >>>>>>>>> >
>> > > >>>>>>>>> > https://github.com/apache/flink-statefun>
>> > > >>>>>>>>> >
>> > > >>>>>>>>> >
>> > > >>>>>>>>> >
>> > > >>>>>>>>> >
>> > > >>>>>>>>> >
>> > > >>>>>>>>>
>> > > >>>>>>>>
>> > >
>> >
>>
>

Re: [DISCUSS] Status of Statefun Project

[Galen Warren via user <us...@flink.apache.org>]

Yes, exactly! And in addition to the base Statefun jars and the jar for the
Java SDK, it does an equivalent copy/register operation for each of the
other SDK libraries (Go, Python, Javascript) so that those libraries are
also available when building the playground examples.

One more question: In order to copy the various build artifacts into the
Docker containers, those artifacts need to be part of the Docker context.
With the playground being a separate project, that's slightly tricky to do,
as there is no guarantee (other than convention) about the relative paths
of *flink-statefun* and* flink-statefun-playground *in someone's local
filesystem. The way I've set this up locally is to copy the playground into
the* flink-statefun* project -- i.e. to *flink-statefun*/playground -- such
that I can set the Docker context to the root folder of *flink-statefun*
and then have access to any local code and/or build artifacts.

I'm wondering if there might be any appetite for making that move
permanent, i.e. moving the playground to *flink-statefun*/playground and
deprecating the standalone playground project. In addition to making the
problem of building with unreleased artifacts a bit simpler to solve, it
would also simplify the process of releasing a new Statefun version, since
the entire process could be handled with a single PR and associated
build/deploy tasks. In other words, a single PR could both update and
deploy the Statefun package and the playground code and images.

As it stands, at least two PRs would be required for each Statefun version
update -- one for *flink-statefun* and one for *flink-statefun-playground*.

Anyway, just an idea. Maybe there's an important reason for these projects
to remain separate. If we do want to keep the playground project where it
is, I could solve the copying problem by requiring the two projects to be
siblings in the file system and by pre-copying the local build artifacts
into a location accessible by the existing Docker contexts. This would
still leave us with the need to have two PRs and releases instead of one,
though.

Thanks for your help!


On Fri, Aug 18, 2023 at 2:45 PM Tzu-Li (Gordon) Tai <tz...@apache.org>
wrote:

> Hi Galen,
>
> > locally built code is copied into the build containers
> so that it can be accessed during the build.
>
> That's exactly what we had been doing for release testing, yes. Sorry I
> missed that detail in my previous response.
>
> And yes, that sounds like a reasonable approach. If I understand you
> correctly, the workflow would become this:
>
>    1. Build the StateFun repo locally to install the snapshot artifact
>    jars + have a local base StateFun image.
>    2. Run the playground in "local" mode, so that it uses the local base
>    StateFun image + builds the playground code using copied artifact jars
>    (instead of pulling from Maven).
>
> That looks good to me!
>
> Thanks,
> Gordon
>
> On Fri, Aug 18, 2023 at 11:33 AM Galen Warren
> <ga...@cvillewarrens.com.invalid> wrote:
>
> > Thanks.
> >
> > If you were to build a local image, as you suggest, how do you access
> that
> > image when building the playground images? All the compilation of
> > playground code happens inside containers, so local images on the host
> > aren't available in those containers. Unless I'm missing something?
> >
> > I've slightly reworked things such that the playground images can be run
> in
> > one of two modes -- the default mode, which works like before, and a
> > "local" mode where locally built code is copied into the build containers
> > so that it can be accessed during the build. It works fine, you just have
> > to define a couple of environment variables when running docker-compose
> to
> > specify default vs. local mode and what versions of Flink and Statefun
> > should be referenced, and then you can build a run the local examples
> > without any additional steps. Does that sound like a reasonable approach?
> >
> >
> > On Fri, Aug 18, 2023 at 2:17 PM Tzu-Li (Gordon) Tai <tzulitai@apache.org
> >
> > wrote:
> >
> > > Hi Galen,
> > >
> > > > Gordon, is there a trick to running the sample code in
> > > flink-statefun-playground against yet-unreleased code that I'm missing?
> > >
> > > You'd have to locally build an image from the release branch, with a
> > > temporary image version tag. Then, in the flink-statefun-playground,
> > change
> > > the image versions in the docker-compose files to use that locally
> built
> > > image. IIRC, that's what we have been doing in the past. Admittedly,
> it's
> > > pretty manual - I don't think the CI manages this workflow.
> > >
> > > Thanks,
> > > Gordon
> > >
> > > On Mon, Aug 14, 2023 at 10:42 AM Galen Warren <galen@cvillewarrens.com
> >
> > > wrote:
> > >
> > > > I created a pull request for this: [FLINK-31619] Upgrade Stateful
> > > > Functions to Flink 1.16.1 by galenwarren · Pull Request #331 ·
> > > > apache/flink-statefun (github.com)
> > > > <https://github.com/apache/flink-statefun/pull/331>.
> > > >
> > > > JIRA is here: [FLINK-31619] Upgrade Stateful Functions to Flink
> 1.16.1
> > -
> > > > ASF JIRA (apache.org)
> > > > <https://issues.apache.org/jira/browse/FLINK-31619?filter=-1>.
> > > >
> > > > Statefun references 1.16.2, despite the title -- that version has
> come
> > > out
> > > > since the issue was created.
> > > >
> > > > I figured out how to run all the playground tests locally, but it
> took
> > a
> > > > bit of reworking of the playground setup with respect to Docker;
> > > > specifically, the Docker contexts used to build the example functions
> > > > needed to be broadened (i.e. moved up the tree) so that, if needed,
> > local
> > > > artifacts/code can be accessed from within the containers at build
> > time.
> > > > Then I made the Docker compose.yml configurable through environment
> > > > variables to allow for them to run in either the original manner --
> > i.e.
> > > > pulling artifacts from public repos -- or in a "local" mode, where
> > > > artifacts are pulled from local builds.
> > > >
> > > > This process is a cleaner if the playground is a subfolder of the
> > > > flink-statefun project rather than be its own repository
> > > > (flink-statefun-playground), because then all the relative paths
> > between
> > > > the playground files and the build artifacts are fixed. So, I'd like
> to
> > > > propose to move the playground files, modified as described above, to
> > > > flink-statefun/playground and retire flink-statefun-playground. I can
> > > > submit separate PR s those changes if everyone is on board.
> > > >
> > > > Also, should I plan to do the same upgrade to handle Flink 1.17.x? It
> > > > should be easy to do, especially while the 1.16.x upgrade is fresh on
> > my
> > > > mind.
> > > >
> > > > Thanks.
> > > >
> > > >
> > > > On Fri, Aug 11, 2023 at 6:40 PM Galen Warren <
> galen@cvillewarrens.com>
> > > > wrote:
> > > >
> > > >> I'm done with the code to make Statefun compatible with Flink 1.16,
> > and
> > > >> all the tests (including e2e succeed). The required changes were
> > pretty
> > > >> minimal.
> > > >>
> > > >> I'm running into a bit of a chicken/egg problem executing the tests
> in
> > > >> flink-statefun-playground
> > > >> <https://github.com/apache/flink-statefun-playground>, though. That
> > > >> project seems to assume that all the various Statefun artifacts are
> > > built
> > > >> and deployed to the various public repositories already. I've looked
> > > into
> > > >> trying to redirect references to local artifacts; however, that's
> also
> > > >> tricky since all the building occurs in Docker containers.
> > > >>
> > > >> Gordon, is there a trick to running the sample code in
> > > >> flink-statefun-playground against yet-unreleased code that I'm
> > missing?
> > > >>
> > > >> Thanks.
> > > >>
> > > >> On Sat, Jun 24, 2023 at 12:40 PM Galen Warren <
> > galen@cvillewarrens.com>
> > > >> wrote:
> > > >>
> > > >>> Great -- thanks!
> > > >>>
> > > >>> I'm going to be out of town for about a week but I'll take a look
> at
> > > >>> this when I'm back.
> > > >>>
> > > >>> On Tue, Jun 20, 2023 at 8:46 AM Martijn Visser <
> mvisser@confluent.io
> > >
> > > >>> wrote:
> > > >>>
> > > >>>> Hi Galen,
> > > >>>>
> > > >>>> Yes, I'll be more than happy to help with Statefun releases.
> > > >>>>
> > > >>>> Best regards,
> > > >>>>
> > > >>>> Martijn
> > > >>>>
> > > >>>> On Tue, Jun 20, 2023 at 2:21 PM Galen Warren <
> > galen@cvillewarrens.com
> > > >
> > > >>>> wrote:
> > > >>>>
> > > >>>>> Thanks.
> > > >>>>>
> > > >>>>> Martijn, to answer your question, I'd need to do a small amount
> of
> > > >>>>> work to get a PR ready, but not much. Happy to do it if we're
> > > deciding to
> > > >>>>> restart Statefun releases -- are we?
> > > >>>>>
> > > >>>>> -- Galen
> > > >>>>>
> > > >>>>> On Sat, Jun 17, 2023 at 9:47 AM Tzu-Li (Gordon) Tai <
> > > >>>>> tzulitai@apache.org> wrote:
> > > >>>>>
> > > >>>>>> > Perhaps he could weigh in on whether the combination of
> > automated
> > > >>>>>> tests plus those smoke tests should be sufficient for testing
> with
> > > new
> > > >>>>>> Flink versions
> > > >>>>>>
> > > >>>>>> What we usually did at the bare minimum for new StateFun
> releases
> > > was
> > > >>>>>> the following:
> > > >>>>>>
> > > >>>>>>    1. Build tests (including the smoke tests in the e2e module,
> > > >>>>>>    which covers important tests like exactly-once verification)
> > > >>>>>>    2. Updating the flink-statefun-playground repo and manually
> > > >>>>>>    running all language examples there.
> > > >>>>>>
> > > >>>>>> If upgrading Flink versions was the only change in the release,
> > I'd
> > > >>>>>> probably say that this is sufficient.
> > > >>>>>>
> > > >>>>>> Best,
> > > >>>>>> Gordon
> > > >>>>>>
> > > >>>>>> On Thu, Jun 15, 2023 at 5:25 AM Martijn Visser <
> > > >>>>>> martijnvisser@apache.org> wrote:
> > > >>>>>>
> > > >>>>>>> Let me know if you have a PR for a Flink update :)
> > > >>>>>>>
> > > >>>>>>> On Thu, Jun 8, 2023 at 5:52 PM Galen Warren via user <
> > > >>>>>>> user@flink.apache.org> wrote:
> > > >>>>>>>
> > > >>>>>>>> Thanks Martijn.
> > > >>>>>>>>
> > > >>>>>>>> Personally, I'm already using a local fork of Statefun that is
> > > >>>>>>>> compatible with Flink 1.16.x, so I wouldn't have any need for
> a
> > > released
> > > >>>>>>>> version compatible with 1.15.x. I'd be happy to do the PRs to
> > > modify
> > > >>>>>>>> Statefun to work with new versions of Flink as they come
> along.
> > > >>>>>>>>
> > > >>>>>>>> As for testing, Statefun does have unit tests and Gordon also
> > sent
> > > >>>>>>>> me instructions a while back for how to do some additional
> smoke
> > > tests
> > > >>>>>>>> which are pretty straightforward. Perhaps he could weigh in on
> > > whether the
> > > >>>>>>>> combination of automated tests plus those smoke tests should
> be
> > > sufficient
> > > >>>>>>>> for testing with new Flink versions (I believe the answer is
> > yes).
> > > >>>>>>>>
> > > >>>>>>>> -- Galen
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> On Thu, Jun 8, 2023 at 8:01 AM Martijn Visser <
> > > >>>>>>>> martijnvisser@apache.org> wrote:
> > > >>>>>>>>
> > > >>>>>>>>> Hi all,
> > > >>>>>>>>>
> > > >>>>>>>>> Apologies for the late reply.
> > > >>>>>>>>>
> > > >>>>>>>>> I'm willing to help out with merging requests in Statefun to
> > keep
> > > >>>>>>>>> them
> > > >>>>>>>>> compatible with new Flink releases and create new releases. I
> > do
> > > >>>>>>>>> think that
> > > >>>>>>>>> validation of the functionality of these releases depends a
> lot
> > > on
> > > >>>>>>>>> those
> > > >>>>>>>>> who do these compatibility updates, with PMC members helping
> > out
> > > >>>>>>>>> with the
> > > >>>>>>>>> formal process.
> > > >>>>>>>>>
> > > >>>>>>>>> > Why can't the Apache Software Foundation allow community
> > > members
> > > >>>>>>>>> to bring
> > > >>>>>>>>> it up to date?
> > > >>>>>>>>>
> > > >>>>>>>>> There's nothing preventing anyone from reviewing any of the
> > > >>>>>>>>> current PRs or
> > > >>>>>>>>> opening new ones. However, none of them are approved [1], so
> > > >>>>>>>>> there's also
> > > >>>>>>>>> nothing to merge.
> > > >>>>>>>>>
> > > >>>>>>>>> > I believe that there are people and companies on this
> mailing
> > > >>>>>>>>> list
> > > >>>>>>>>> interested in supporting Apache Flink Stateful Functions.
> > > >>>>>>>>>
> > > >>>>>>>>> If so, then now is the time to show.
> > > >>>>>>>>>
> > > >>>>>>>>> Would there be a preference to create a release with Galen's
> > > merged
> > > >>>>>>>>> compatibility update to Flink 1.15.2, or do we want to skip
> > that
> > > >>>>>>>>> and go
> > > >>>>>>>>> straight to a newer version?
> > > >>>>>>>>>
> > > >>>>>>>>> Best regards,
> > > >>>>>>>>>
> > > >>>>>>>>> Martijn
> > > >>>>>>>>>
> > > >>>>>>>>> [1]
> > > >>>>>>>>>
> > > >>>>>>>>>
> > >
> >
> https://github.com/apache/flink-statefun/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved
> > > >>>>>>>>>
> > > >>>>>>>>> On Tue, Jun 6, 2023 at 3:55 PM Marco Villalobos <
> > > >>>>>>>>> mvillalobos@kineteque.com>
> > > >>>>>>>>> wrote:
> > > >>>>>>>>>
> > > >>>>>>>>> > Why can't the Apache Software Foundation allow community
> > > members
> > > >>>>>>>>> to bring
> > > >>>>>>>>> > it up to date?
> > > >>>>>>>>> >
> > > >>>>>>>>> > What's the process for that?
> > > >>>>>>>>> >
> > > >>>>>>>>> > I believe that there are people and companies on this
> mailing
> > > >>>>>>>>> list
> > > >>>>>>>>> > interested in supporting Apache Flink Stateful Functions.
> > > >>>>>>>>> >
> > > >>>>>>>>> > You already had two people on this thread express interest.
> > > >>>>>>>>> >
> > > >>>>>>>>> > At the very least, we could keep the library versions up to
> > > date.
> > > >>>>>>>>> >
> > > >>>>>>>>> > There are only a small list of new features that might be
> > > >>>>>>>>> worthwhile:
> > > >>>>>>>>> >
> > > >>>>>>>>> > 1. event time processing
> > > >>>>>>>>> > 2. state rest api
> > > >>>>>>>>> >
> > > >>>>>>>>> >
> > > >>>>>>>>> > On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <
> > > chesnay@apache.org>
> > > >>>>>>>>> wrote:
> > > >>>>>>>>> >
> > > >>>>>>>>> > If you were to fork it *and want to redistribute it* then
> the
> > > >>>>>>>>> short
> > > >>>>>>>>> > version is that
> > > >>>>>>>>> >
> > > >>>>>>>>> >    1. you have to adhere to the Apache licensing
> requirements
> > > >>>>>>>>> >    2. you have to make it clear that your fork does not
> > belong
> > > >>>>>>>>> to the
> > > >>>>>>>>> >    Apache Flink project. (Trademarks and all that)
> > > >>>>>>>>> >
> > > >>>>>>>>> > Neither should be significant hurdles (there should also be
> > > >>>>>>>>> plenty of
> > > >>>>>>>>> > online resources regarding 1), and if you do this then you
> > can
> > > >>>>>>>>> freely share
> > > >>>>>>>>> > your fork with others.
> > > >>>>>>>>> >
> > > >>>>>>>>> > I've also pinged Martijn to take a look at this thread.
> > > >>>>>>>>> > To my knowledge the project hasn't decided anything yet.
> > > >>>>>>>>> >
> > > >>>>>>>>> > On 27/05/2023 04:05, Galen Warren wrote:
> > > >>>>>>>>> >
> > > >>>>>>>>> > Ok, I get it. No interest.
> > > >>>>>>>>> >
> > > >>>>>>>>> > If this project is being abandoned, I guess I'll work with
> my
> > > >>>>>>>>> own fork. Is
> > > >>>>>>>>> > there anything I should consider here? Can I share it with
> > > other
> > > >>>>>>>>> people who
> > > >>>>>>>>> > use this project?
> > > >>>>>>>>> >
> > > >>>>>>>>> > On Tue, May 16, 2023 at 10:50 AM Galen Warren <
> > > >>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
> > > >>>>>>>>> > wrote:
> > > >>>>>>>>> >
> > > >>>>>>>>> >
> > > >>>>>>>>> > Hi Martijn, since you opened this discussion thread, I'm
> > > curious
> > > >>>>>>>>> what your
> > > >>>>>>>>> > thoughts are in light of the responses? Thanks.
> > > >>>>>>>>> >
> > > >>>>>>>>> > On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <
> > > >>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
> > > >>>>>>>>> > wrote:
> > > >>>>>>>>> >
> > > >>>>>>>>> >
> > > >>>>>>>>> > I use Apache Flink for stream processing, and StateFun as a
> > > >>>>>>>>> hand-off
> > > >>>>>>>>> >
> > > >>>>>>>>> > point for the rest of the application.
> > > >>>>>>>>> > It serves well as a bridge between a Flink Streaming job
> and
> > > >>>>>>>>> > micro-services.
> > > >>>>>>>>> >
> > > >>>>>>>>> > This is essentially how I use it as well, and I would also
> be
> > > >>>>>>>>> sad to see
> > > >>>>>>>>> > it sunsetted. It works well; I don't know that there is a
> lot
> > > of
> > > >>>>>>>>> new
> > > >>>>>>>>> > development required, but if there are no new Statefun
> > > releases,
> > > >>>>>>>>> then
> > > >>>>>>>>> > Statefun can only be used with older Flink versions.
> > > >>>>>>>>> >
> > > >>>>>>>>> > On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
> > > >>>>>>>>> mvillalobos@kineteque.com> wrote:
> > > >>>>>>>>> >
> > > >>>>>>>>> >
> > > >>>>>>>>> > I am currently using Stateful Functions in my application.
> > > >>>>>>>>> >
> > > >>>>>>>>> > I use Apache Flink for stream processing, and StateFun as a
> > > >>>>>>>>> hand-off
> > > >>>>>>>>> > point for the rest of the application.
> > > >>>>>>>>> > It serves well as a bridge between a Flink Streaming job
> and
> > > >>>>>>>>> > micro-services.
> > > >>>>>>>>> >
> > > >>>>>>>>> > I would be disappointed if StateFun was sunsetted.  Its a
> > good
> > > >>>>>>>>> idea.
> > > >>>>>>>>> >
> > > >>>>>>>>> > If there is anything I can do to help, as a contributor
> > > perhaps,
> > > >>>>>>>>> please
> > > >>>>>>>>> > let me know.
> > > >>>>>>>>> >
> > > >>>>>>>>> >
> > > >>>>>>>>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <
> > > >>>>>>>>> martijnvisser@apache.org> <ma...@apache.org>
> > > >>>>>>>>> >
> > > >>>>>>>>> > wrote:
> > > >>>>>>>>> >
> > > >>>>>>>>> > Hi everyone,
> > > >>>>>>>>> >
> > > >>>>>>>>> > I want to open a discussion on the status of the Statefun
> > > >>>>>>>>> Project [1]
> > > >>>>>>>>> >
> > > >>>>>>>>> > in Apache Flink. As you might have noticed, there hasn't
> been
> > > >>>>>>>>> much
> > > >>>>>>>>> > development over the past months in the Statefun repository
> > > [2].
> > > >>>>>>>>> There is
> > > >>>>>>>>> > currently a lack of active contributors and committers who
> > are
> > > >>>>>>>>> able to help
> > > >>>>>>>>> > with the maintenance of the project.
> > > >>>>>>>>> >
> > > >>>>>>>>> > In order to improve the situation, we need to solve the
> lack
> > of
> > > >>>>>>>>> >
> > > >>>>>>>>> > committers and the lack of contributors.
> > > >>>>>>>>> >
> > > >>>>>>>>> > On the lack of committers:
> > > >>>>>>>>> >
> > > >>>>>>>>> > 1. Ideally, there are some of the current Flink committers
> > who
> > > >>>>>>>>> have
> > > >>>>>>>>> >
> > > >>>>>>>>> > the bandwidth and can help with reviewing PRs and merging
> > them.
> > > >>>>>>>>> >
> > > >>>>>>>>> > 2. If that's not an option, it could be a consideration
> that
> > > >>>>>>>>> current
> > > >>>>>>>>> >
> > > >>>>>>>>> > committers only approve and review PRs, that are approved
> by
> > > >>>>>>>>> those who are
> > > >>>>>>>>> > willing to contribute to Statefun and if the CI passes
> > > >>>>>>>>> >
> > > >>>>>>>>> > On the lack of contributors:
> > > >>>>>>>>> >
> > > >>>>>>>>> > 3. Next to having this discussion on the Dev and User
> mailing
> > > >>>>>>>>> list, we
> > > >>>>>>>>> >
> > > >>>>>>>>> > can also create a blog with a call for new contributors on
> > the
> > > >>>>>>>>> Flink
> > > >>>>>>>>> > project website, send out some tweets on the Flink /
> Statefun
> > > >>>>>>>>> twitter
> > > >>>>>>>>> > accounts, post messages on Slack etc. In that message, we
> > would
> > > >>>>>>>>> inform how
> > > >>>>>>>>> > those that are interested in contributing can start and
> where
> > > >>>>>>>>> they could
> > > >>>>>>>>> > reach out for more information.
> > > >>>>>>>>> >
> > > >>>>>>>>> > There's also option 4. where a group of interested people
> > would
> > > >>>>>>>>> split
> > > >>>>>>>>> >
> > > >>>>>>>>> > Statefun from the Flink project and make it a separate top
> > > level
> > > >>>>>>>>> project
> > > >>>>>>>>> > under the Apache Flink umbrella (similar as recently has
> > > >>>>>>>>> happened with
> > > >>>>>>>>> > Flink Table Store, which has become Apache Paimon).
> > > >>>>>>>>> >
> > > >>>>>>>>> > If we see no improvements in the coming period, we should
> > > >>>>>>>>> consider
> > > >>>>>>>>> >
> > > >>>>>>>>> > sunsetting Statefun and communicate that clearly to the
> > users.
> > > >>>>>>>>> >
> > > >>>>>>>>> > I'm looking forward to your thoughts.
> > > >>>>>>>>> >
> > > >>>>>>>>> > Best regards,
> > > >>>>>>>>> >
> > > >>>>>>>>> > Martijn
> > > >>>>>>>>> >
> > > >>>>>>>>> > [1]
> > > >>>>>>>>>
> https://nightlies.apache.org/flink/flink-statefun-docs-master/
> > <
> > > >>>>>>>>> >
> > > >>>>>>>>> >
> > https://nightlies.apache.org/flink/flink-statefun-docs-master/
> > > >
> > > >>>>>>>>> >
> > > >>>>>>>>> > [2] https://github.com/apache/flink-statefun <
> > > >>>>>>>>> >
> > > >>>>>>>>> > https://github.com/apache/flink-statefun>
> > > >>>>>>>>> >
> > > >>>>>>>>> >
> > > >>>>>>>>> >
> > > >>>>>>>>> >
> > > >>>>>>>>> >
> > > >>>>>>>>>
> > > >>>>>>>>
> > >
> >
>

Re: [DISCUSS] Status of Statefun Project

[Galen Warren <ga...@cvillewarrens.com.INVALID>]

Yes, exactly! And in addition to the base Statefun jars and the jar for the
Java SDK, it does an equivalent copy/register operation for each of the
other SDK libraries (Go, Python, Javascript) so that those libraries are
also available when building the playground examples.

One more question: In order to copy the various build artifacts into the
Docker containers, those artifacts need to be part of the Docker context.
With the playground being a separate project, that's slightly tricky to do,
as there is no guarantee (other than convention) about the relative paths
of *flink-statefun* and* flink-statefun-playground *in someone's local
filesystem. The way I've set this up locally is to copy the playground into
the* flink-statefun* project -- i.e. to *flink-statefun*/playground -- such
that I can set the Docker context to the root folder of *flink-statefun*
and then have access to any local code and/or build artifacts.

I'm wondering if there might be any appetite for making that move
permanent, i.e. moving the playground to *flink-statefun*/playground and
deprecating the standalone playground project. In addition to making the
problem of building with unreleased artifacts a bit simpler to solve, it
would also simplify the process of releasing a new Statefun version, since
the entire process could be handled with a single PR and associated
build/deploy tasks. In other words, a single PR could both update and
deploy the Statefun package and the playground code and images.

As it stands, at least two PRs would be required for each Statefun version
update -- one for *flink-statefun* and one for *flink-statefun-playground*.

Anyway, just an idea. Maybe there's an important reason for these projects
to remain separate. If we do want to keep the playground project where it
is, I could solve the copying problem by requiring the two projects to be
siblings in the file system and by pre-copying the local build artifacts
into a location accessible by the existing Docker contexts. This would
still leave us with the need to have two PRs and releases instead of one,
though.

Thanks for your help!


On Fri, Aug 18, 2023 at 2:45 PM Tzu-Li (Gordon) Tai <tz...@apache.org>
wrote:

> Hi Galen,
>
> > locally built code is copied into the build containers
> so that it can be accessed during the build.
>
> That's exactly what we had been doing for release testing, yes. Sorry I
> missed that detail in my previous response.
>
> And yes, that sounds like a reasonable approach. If I understand you
> correctly, the workflow would become this:
>
>    1. Build the StateFun repo locally to install the snapshot artifact
>    jars + have a local base StateFun image.
>    2. Run the playground in "local" mode, so that it uses the local base
>    StateFun image + builds the playground code using copied artifact jars
>    (instead of pulling from Maven).
>
> That looks good to me!
>
> Thanks,
> Gordon
>
> On Fri, Aug 18, 2023 at 11:33 AM Galen Warren
> <ga...@cvillewarrens.com.invalid> wrote:
>
> > Thanks.
> >
> > If you were to build a local image, as you suggest, how do you access
> that
> > image when building the playground images? All the compilation of
> > playground code happens inside containers, so local images on the host
> > aren't available in those containers. Unless I'm missing something?
> >
> > I've slightly reworked things such that the playground images can be run
> in
> > one of two modes -- the default mode, which works like before, and a
> > "local" mode where locally built code is copied into the build containers
> > so that it can be accessed during the build. It works fine, you just have
> > to define a couple of environment variables when running docker-compose
> to
> > specify default vs. local mode and what versions of Flink and Statefun
> > should be referenced, and then you can build a run the local examples
> > without any additional steps. Does that sound like a reasonable approach?
> >
> >
> > On Fri, Aug 18, 2023 at 2:17 PM Tzu-Li (Gordon) Tai <tzulitai@apache.org
> >
> > wrote:
> >
> > > Hi Galen,
> > >
> > > > Gordon, is there a trick to running the sample code in
> > > flink-statefun-playground against yet-unreleased code that I'm missing?
> > >
> > > You'd have to locally build an image from the release branch, with a
> > > temporary image version tag. Then, in the flink-statefun-playground,
> > change
> > > the image versions in the docker-compose files to use that locally
> built
> > > image. IIRC, that's what we have been doing in the past. Admittedly,
> it's
> > > pretty manual - I don't think the CI manages this workflow.
> > >
> > > Thanks,
> > > Gordon
> > >
> > > On Mon, Aug 14, 2023 at 10:42 AM Galen Warren <galen@cvillewarrens.com
> >
> > > wrote:
> > >
> > > > I created a pull request for this: [FLINK-31619] Upgrade Stateful
> > > > Functions to Flink 1.16.1 by galenwarren · Pull Request #331 ·
> > > > apache/flink-statefun (github.com)
> > > > <https://github.com/apache/flink-statefun/pull/331>.
> > > >
> > > > JIRA is here: [FLINK-31619] Upgrade Stateful Functions to Flink
> 1.16.1
> > -
> > > > ASF JIRA (apache.org)
> > > > <https://issues.apache.org/jira/browse/FLINK-31619?filter=-1>.
> > > >
> > > > Statefun references 1.16.2, despite the title -- that version has
> come
> > > out
> > > > since the issue was created.
> > > >
> > > > I figured out how to run all the playground tests locally, but it
> took
> > a
> > > > bit of reworking of the playground setup with respect to Docker;
> > > > specifically, the Docker contexts used to build the example functions
> > > > needed to be broadened (i.e. moved up the tree) so that, if needed,
> > local
> > > > artifacts/code can be accessed from within the containers at build
> > time.
> > > > Then I made the Docker compose.yml configurable through environment
> > > > variables to allow for them to run in either the original manner --
> > i.e.
> > > > pulling artifacts from public repos -- or in a "local" mode, where
> > > > artifacts are pulled from local builds.
> > > >
> > > > This process is a cleaner if the playground is a subfolder of the
> > > > flink-statefun project rather than be its own repository
> > > > (flink-statefun-playground), because then all the relative paths
> > between
> > > > the playground files and the build artifacts are fixed. So, I'd like
> to
> > > > propose to move the playground files, modified as described above, to
> > > > flink-statefun/playground and retire flink-statefun-playground. I can
> > > > submit separate PR s those changes if everyone is on board.
> > > >
> > > > Also, should I plan to do the same upgrade to handle Flink 1.17.x? It
> > > > should be easy to do, especially while the 1.16.x upgrade is fresh on
> > my
> > > > mind.
> > > >
> > > > Thanks.
> > > >
> > > >
> > > > On Fri, Aug 11, 2023 at 6:40 PM Galen Warren <
> galen@cvillewarrens.com>
> > > > wrote:
> > > >
> > > >> I'm done with the code to make Statefun compatible with Flink 1.16,
> > and
> > > >> all the tests (including e2e succeed). The required changes were
> > pretty
> > > >> minimal.
> > > >>
> > > >> I'm running into a bit of a chicken/egg problem executing the tests
> in
> > > >> flink-statefun-playground
> > > >> <https://github.com/apache/flink-statefun-playground>, though. That
> > > >> project seems to assume that all the various Statefun artifacts are
> > > built
> > > >> and deployed to the various public repositories already. I've looked
> > > into
> > > >> trying to redirect references to local artifacts; however, that's
> also
> > > >> tricky since all the building occurs in Docker containers.
> > > >>
> > > >> Gordon, is there a trick to running the sample code in
> > > >> flink-statefun-playground against yet-unreleased code that I'm
> > missing?
> > > >>
> > > >> Thanks.
> > > >>
> > > >> On Sat, Jun 24, 2023 at 12:40 PM Galen Warren <
> > galen@cvillewarrens.com>
> > > >> wrote:
> > > >>
> > > >>> Great -- thanks!
> > > >>>
> > > >>> I'm going to be out of town for about a week but I'll take a look
> at
> > > >>> this when I'm back.
> > > >>>
> > > >>> On Tue, Jun 20, 2023 at 8:46 AM Martijn Visser <
> mvisser@confluent.io
> > >
> > > >>> wrote:
> > > >>>
> > > >>>> Hi Galen,
> > > >>>>
> > > >>>> Yes, I'll be more than happy to help with Statefun releases.
> > > >>>>
> > > >>>> Best regards,
> > > >>>>
> > > >>>> Martijn
> > > >>>>
> > > >>>> On Tue, Jun 20, 2023 at 2:21 PM Galen Warren <
> > galen@cvillewarrens.com
> > > >
> > > >>>> wrote:
> > > >>>>
> > > >>>>> Thanks.
> > > >>>>>
> > > >>>>> Martijn, to answer your question, I'd need to do a small amount
> of
> > > >>>>> work to get a PR ready, but not much. Happy to do it if we're
> > > deciding to
> > > >>>>> restart Statefun releases -- are we?
> > > >>>>>
> > > >>>>> -- Galen
> > > >>>>>
> > > >>>>> On Sat, Jun 17, 2023 at 9:47 AM Tzu-Li (Gordon) Tai <
> > > >>>>> tzulitai@apache.org> wrote:
> > > >>>>>
> > > >>>>>> > Perhaps he could weigh in on whether the combination of
> > automated
> > > >>>>>> tests plus those smoke tests should be sufficient for testing
> with
> > > new
> > > >>>>>> Flink versions
> > > >>>>>>
> > > >>>>>> What we usually did at the bare minimum for new StateFun
> releases
> > > was
> > > >>>>>> the following:
> > > >>>>>>
> > > >>>>>>    1. Build tests (including the smoke tests in the e2e module,
> > > >>>>>>    which covers important tests like exactly-once verification)
> > > >>>>>>    2. Updating the flink-statefun-playground repo and manually
> > > >>>>>>    running all language examples there.
> > > >>>>>>
> > > >>>>>> If upgrading Flink versions was the only change in the release,
> > I'd
> > > >>>>>> probably say that this is sufficient.
> > > >>>>>>
> > > >>>>>> Best,
> > > >>>>>> Gordon
> > > >>>>>>
> > > >>>>>> On Thu, Jun 15, 2023 at 5:25 AM Martijn Visser <
> > > >>>>>> martijnvisser@apache.org> wrote:
> > > >>>>>>
> > > >>>>>>> Let me know if you have a PR for a Flink update :)
> > > >>>>>>>
> > > >>>>>>> On Thu, Jun 8, 2023 at 5:52 PM Galen Warren via user <
> > > >>>>>>> user@flink.apache.org> wrote:
> > > >>>>>>>
> > > >>>>>>>> Thanks Martijn.
> > > >>>>>>>>
> > > >>>>>>>> Personally, I'm already using a local fork of Statefun that is
> > > >>>>>>>> compatible with Flink 1.16.x, so I wouldn't have any need for
> a
> > > released
> > > >>>>>>>> version compatible with 1.15.x. I'd be happy to do the PRs to
> > > modify
> > > >>>>>>>> Statefun to work with new versions of Flink as they come
> along.
> > > >>>>>>>>
> > > >>>>>>>> As for testing, Statefun does have unit tests and Gordon also
> > sent
> > > >>>>>>>> me instructions a while back for how to do some additional
> smoke
> > > tests
> > > >>>>>>>> which are pretty straightforward. Perhaps he could weigh in on
> > > whether the
> > > >>>>>>>> combination of automated tests plus those smoke tests should
> be
> > > sufficient
> > > >>>>>>>> for testing with new Flink versions (I believe the answer is
> > yes).
> > > >>>>>>>>
> > > >>>>>>>> -- Galen
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> On Thu, Jun 8, 2023 at 8:01 AM Martijn Visser <
> > > >>>>>>>> martijnvisser@apache.org> wrote:
> > > >>>>>>>>
> > > >>>>>>>>> Hi all,
> > > >>>>>>>>>
> > > >>>>>>>>> Apologies for the late reply.
> > > >>>>>>>>>
> > > >>>>>>>>> I'm willing to help out with merging requests in Statefun to
> > keep
> > > >>>>>>>>> them
> > > >>>>>>>>> compatible with new Flink releases and create new releases. I
> > do
> > > >>>>>>>>> think that
> > > >>>>>>>>> validation of the functionality of these releases depends a
> lot
> > > on
> > > >>>>>>>>> those
> > > >>>>>>>>> who do these compatibility updates, with PMC members helping
> > out
> > > >>>>>>>>> with the
> > > >>>>>>>>> formal process.
> > > >>>>>>>>>
> > > >>>>>>>>> > Why can't the Apache Software Foundation allow community
> > > members
> > > >>>>>>>>> to bring
> > > >>>>>>>>> it up to date?
> > > >>>>>>>>>
> > > >>>>>>>>> There's nothing preventing anyone from reviewing any of the
> > > >>>>>>>>> current PRs or
> > > >>>>>>>>> opening new ones. However, none of them are approved [1], so
> > > >>>>>>>>> there's also
> > > >>>>>>>>> nothing to merge.
> > > >>>>>>>>>
> > > >>>>>>>>> > I believe that there are people and companies on this
> mailing
> > > >>>>>>>>> list
> > > >>>>>>>>> interested in supporting Apache Flink Stateful Functions.
> > > >>>>>>>>>
> > > >>>>>>>>> If so, then now is the time to show.
> > > >>>>>>>>>
> > > >>>>>>>>> Would there be a preference to create a release with Galen's
> > > merged
> > > >>>>>>>>> compatibility update to Flink 1.15.2, or do we want to skip
> > that
> > > >>>>>>>>> and go
> > > >>>>>>>>> straight to a newer version?
> > > >>>>>>>>>
> > > >>>>>>>>> Best regards,
> > > >>>>>>>>>
> > > >>>>>>>>> Martijn
> > > >>>>>>>>>
> > > >>>>>>>>> [1]
> > > >>>>>>>>>
> > > >>>>>>>>>
> > >
> >
> https://github.com/apache/flink-statefun/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved
> > > >>>>>>>>>
> > > >>>>>>>>> On Tue, Jun 6, 2023 at 3:55 PM Marco Villalobos <
> > > >>>>>>>>> mvillalobos@kineteque.com>
> > > >>>>>>>>> wrote:
> > > >>>>>>>>>
> > > >>>>>>>>> > Why can't the Apache Software Foundation allow community
> > > members
> > > >>>>>>>>> to bring
> > > >>>>>>>>> > it up to date?
> > > >>>>>>>>> >
> > > >>>>>>>>> > What's the process for that?
> > > >>>>>>>>> >
> > > >>>>>>>>> > I believe that there are people and companies on this
> mailing
> > > >>>>>>>>> list
> > > >>>>>>>>> > interested in supporting Apache Flink Stateful Functions.
> > > >>>>>>>>> >
> > > >>>>>>>>> > You already had two people on this thread express interest.
> > > >>>>>>>>> >
> > > >>>>>>>>> > At the very least, we could keep the library versions up to
> > > date.
> > > >>>>>>>>> >
> > > >>>>>>>>> > There are only a small list of new features that might be
> > > >>>>>>>>> worthwhile:
> > > >>>>>>>>> >
> > > >>>>>>>>> > 1. event time processing
> > > >>>>>>>>> > 2. state rest api
> > > >>>>>>>>> >
> > > >>>>>>>>> >
> > > >>>>>>>>> > On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <
> > > chesnay@apache.org>
> > > >>>>>>>>> wrote:
> > > >>>>>>>>> >
> > > >>>>>>>>> > If you were to fork it *and want to redistribute it* then
> the
> > > >>>>>>>>> short
> > > >>>>>>>>> > version is that
> > > >>>>>>>>> >
> > > >>>>>>>>> >    1. you have to adhere to the Apache licensing
> requirements
> > > >>>>>>>>> >    2. you have to make it clear that your fork does not
> > belong
> > > >>>>>>>>> to the
> > > >>>>>>>>> >    Apache Flink project. (Trademarks and all that)
> > > >>>>>>>>> >
> > > >>>>>>>>> > Neither should be significant hurdles (there should also be
> > > >>>>>>>>> plenty of
> > > >>>>>>>>> > online resources regarding 1), and if you do this then you
> > can
> > > >>>>>>>>> freely share
> > > >>>>>>>>> > your fork with others.
> > > >>>>>>>>> >
> > > >>>>>>>>> > I've also pinged Martijn to take a look at this thread.
> > > >>>>>>>>> > To my knowledge the project hasn't decided anything yet.
> > > >>>>>>>>> >
> > > >>>>>>>>> > On 27/05/2023 04:05, Galen Warren wrote:
> > > >>>>>>>>> >
> > > >>>>>>>>> > Ok, I get it. No interest.
> > > >>>>>>>>> >
> > > >>>>>>>>> > If this project is being abandoned, I guess I'll work with
> my
> > > >>>>>>>>> own fork. Is
> > > >>>>>>>>> > there anything I should consider here? Can I share it with
> > > other
> > > >>>>>>>>> people who
> > > >>>>>>>>> > use this project?
> > > >>>>>>>>> >
> > > >>>>>>>>> > On Tue, May 16, 2023 at 10:50 AM Galen Warren <
> > > >>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
> > > >>>>>>>>> > wrote:
> > > >>>>>>>>> >
> > > >>>>>>>>> >
> > > >>>>>>>>> > Hi Martijn, since you opened this discussion thread, I'm
> > > curious
> > > >>>>>>>>> what your
> > > >>>>>>>>> > thoughts are in light of the responses? Thanks.
> > > >>>>>>>>> >
> > > >>>>>>>>> > On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <
> > > >>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
> > > >>>>>>>>> > wrote:
> > > >>>>>>>>> >
> > > >>>>>>>>> >
> > > >>>>>>>>> > I use Apache Flink for stream processing, and StateFun as a
> > > >>>>>>>>> hand-off
> > > >>>>>>>>> >
> > > >>>>>>>>> > point for the rest of the application.
> > > >>>>>>>>> > It serves well as a bridge between a Flink Streaming job
> and
> > > >>>>>>>>> > micro-services.
> > > >>>>>>>>> >
> > > >>>>>>>>> > This is essentially how I use it as well, and I would also
> be
> > > >>>>>>>>> sad to see
> > > >>>>>>>>> > it sunsetted. It works well; I don't know that there is a
> lot
> > > of
> > > >>>>>>>>> new
> > > >>>>>>>>> > development required, but if there are no new Statefun
> > > releases,
> > > >>>>>>>>> then
> > > >>>>>>>>> > Statefun can only be used with older Flink versions.
> > > >>>>>>>>> >
> > > >>>>>>>>> > On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
> > > >>>>>>>>> mvillalobos@kineteque.com> wrote:
> > > >>>>>>>>> >
> > > >>>>>>>>> >
> > > >>>>>>>>> > I am currently using Stateful Functions in my application.
> > > >>>>>>>>> >
> > > >>>>>>>>> > I use Apache Flink for stream processing, and StateFun as a
> > > >>>>>>>>> hand-off
> > > >>>>>>>>> > point for the rest of the application.
> > > >>>>>>>>> > It serves well as a bridge between a Flink Streaming job
> and
> > > >>>>>>>>> > micro-services.
> > > >>>>>>>>> >
> > > >>>>>>>>> > I would be disappointed if StateFun was sunsetted.  Its a
> > good
> > > >>>>>>>>> idea.
> > > >>>>>>>>> >
> > > >>>>>>>>> > If there is anything I can do to help, as a contributor
> > > perhaps,
> > > >>>>>>>>> please
> > > >>>>>>>>> > let me know.
> > > >>>>>>>>> >
> > > >>>>>>>>> >
> > > >>>>>>>>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <
> > > >>>>>>>>> martijnvisser@apache.org> <ma...@apache.org>
> > > >>>>>>>>> >
> > > >>>>>>>>> > wrote:
> > > >>>>>>>>> >
> > > >>>>>>>>> > Hi everyone,
> > > >>>>>>>>> >
> > > >>>>>>>>> > I want to open a discussion on the status of the Statefun
> > > >>>>>>>>> Project [1]
> > > >>>>>>>>> >
> > > >>>>>>>>> > in Apache Flink. As you might have noticed, there hasn't
> been
> > > >>>>>>>>> much
> > > >>>>>>>>> > development over the past months in the Statefun repository
> > > [2].
> > > >>>>>>>>> There is
> > > >>>>>>>>> > currently a lack of active contributors and committers who
> > are
> > > >>>>>>>>> able to help
> > > >>>>>>>>> > with the maintenance of the project.
> > > >>>>>>>>> >
> > > >>>>>>>>> > In order to improve the situation, we need to solve the
> lack
> > of
> > > >>>>>>>>> >
> > > >>>>>>>>> > committers and the lack of contributors.
> > > >>>>>>>>> >
> > > >>>>>>>>> > On the lack of committers:
> > > >>>>>>>>> >
> > > >>>>>>>>> > 1. Ideally, there are some of the current Flink committers
> > who
> > > >>>>>>>>> have
> > > >>>>>>>>> >
> > > >>>>>>>>> > the bandwidth and can help with reviewing PRs and merging
> > them.
> > > >>>>>>>>> >
> > > >>>>>>>>> > 2. If that's not an option, it could be a consideration
> that
> > > >>>>>>>>> current
> > > >>>>>>>>> >
> > > >>>>>>>>> > committers only approve and review PRs, that are approved
> by
> > > >>>>>>>>> those who are
> > > >>>>>>>>> > willing to contribute to Statefun and if the CI passes
> > > >>>>>>>>> >
> > > >>>>>>>>> > On the lack of contributors:
> > > >>>>>>>>> >
> > > >>>>>>>>> > 3. Next to having this discussion on the Dev and User
> mailing
> > > >>>>>>>>> list, we
> > > >>>>>>>>> >
> > > >>>>>>>>> > can also create a blog with a call for new contributors on
> > the
> > > >>>>>>>>> Flink
> > > >>>>>>>>> > project website, send out some tweets on the Flink /
> Statefun
> > > >>>>>>>>> twitter
> > > >>>>>>>>> > accounts, post messages on Slack etc. In that message, we
> > would
> > > >>>>>>>>> inform how
> > > >>>>>>>>> > those that are interested in contributing can start and
> where
> > > >>>>>>>>> they could
> > > >>>>>>>>> > reach out for more information.
> > > >>>>>>>>> >
> > > >>>>>>>>> > There's also option 4. where a group of interested people
> > would
> > > >>>>>>>>> split
> > > >>>>>>>>> >
> > > >>>>>>>>> > Statefun from the Flink project and make it a separate top
> > > level
> > > >>>>>>>>> project
> > > >>>>>>>>> > under the Apache Flink umbrella (similar as recently has
> > > >>>>>>>>> happened with
> > > >>>>>>>>> > Flink Table Store, which has become Apache Paimon).
> > > >>>>>>>>> >
> > > >>>>>>>>> > If we see no improvements in the coming period, we should
> > > >>>>>>>>> consider
> > > >>>>>>>>> >
> > > >>>>>>>>> > sunsetting Statefun and communicate that clearly to the
> > users.
> > > >>>>>>>>> >
> > > >>>>>>>>> > I'm looking forward to your thoughts.
> > > >>>>>>>>> >
> > > >>>>>>>>> > Best regards,
> > > >>>>>>>>> >
> > > >>>>>>>>> > Martijn
> > > >>>>>>>>> >
> > > >>>>>>>>> > [1]
> > > >>>>>>>>>
> https://nightlies.apache.org/flink/flink-statefun-docs-master/
> > <
> > > >>>>>>>>> >
> > > >>>>>>>>> >
> > https://nightlies.apache.org/flink/flink-statefun-docs-master/
> > > >
> > > >>>>>>>>> >
> > > >>>>>>>>> > [2] https://github.com/apache/flink-statefun <
> > > >>>>>>>>> >
> > > >>>>>>>>> > https://github.com/apache/flink-statefun>
> > > >>>>>>>>> >
> > > >>>>>>>>> >
> > > >>>>>>>>> >
> > > >>>>>>>>> >
> > > >>>>>>>>> >
> > > >>>>>>>>>
> > > >>>>>>>>
> > >
> >
>

Re: [DISCUSS] Status of Statefun Project

["Tzu-Li (Gordon) Tai" <tz...@apache.org>]

Hi Galen,

> locally built code is copied into the build containers
so that it can be accessed during the build.

That's exactly what we had been doing for release testing, yes. Sorry I
missed that detail in my previous response.

And yes, that sounds like a reasonable approach. If I understand you
correctly, the workflow would become this:

   1. Build the StateFun repo locally to install the snapshot artifact
   jars + have a local base StateFun image.
   2. Run the playground in "local" mode, so that it uses the local base
   StateFun image + builds the playground code using copied artifact jars
   (instead of pulling from Maven).

That looks good to me!

Thanks,
Gordon

On Fri, Aug 18, 2023 at 11:33 AM Galen Warren
<ga...@cvillewarrens.com.invalid> wrote:

> Thanks.
>
> If you were to build a local image, as you suggest, how do you access that
> image when building the playground images? All the compilation of
> playground code happens inside containers, so local images on the host
> aren't available in those containers. Unless I'm missing something?
>
> I've slightly reworked things such that the playground images can be run in
> one of two modes -- the default mode, which works like before, and a
> "local" mode where locally built code is copied into the build containers
> so that it can be accessed during the build. It works fine, you just have
> to define a couple of environment variables when running docker-compose to
> specify default vs. local mode and what versions of Flink and Statefun
> should be referenced, and then you can build a run the local examples
> without any additional steps. Does that sound like a reasonable approach?
>
>
> On Fri, Aug 18, 2023 at 2:17 PM Tzu-Li (Gordon) Tai <tz...@apache.org>
> wrote:
>
> > Hi Galen,
> >
> > > Gordon, is there a trick to running the sample code in
> > flink-statefun-playground against yet-unreleased code that I'm missing?
> >
> > You'd have to locally build an image from the release branch, with a
> > temporary image version tag. Then, in the flink-statefun-playground,
> change
> > the image versions in the docker-compose files to use that locally built
> > image. IIRC, that's what we have been doing in the past. Admittedly, it's
> > pretty manual - I don't think the CI manages this workflow.
> >
> > Thanks,
> > Gordon
> >
> > On Mon, Aug 14, 2023 at 10:42 AM Galen Warren <ga...@cvillewarrens.com>
> > wrote:
> >
> > > I created a pull request for this: [FLINK-31619] Upgrade Stateful
> > > Functions to Flink 1.16.1 by galenwarren · Pull Request #331 ·
> > > apache/flink-statefun (github.com)
> > > <https://github.com/apache/flink-statefun/pull/331>.
> > >
> > > JIRA is here: [FLINK-31619] Upgrade Stateful Functions to Flink 1.16.1
> -
> > > ASF JIRA (apache.org)
> > > <https://issues.apache.org/jira/browse/FLINK-31619?filter=-1>.
> > >
> > > Statefun references 1.16.2, despite the title -- that version has come
> > out
> > > since the issue was created.
> > >
> > > I figured out how to run all the playground tests locally, but it took
> a
> > > bit of reworking of the playground setup with respect to Docker;
> > > specifically, the Docker contexts used to build the example functions
> > > needed to be broadened (i.e. moved up the tree) so that, if needed,
> local
> > > artifacts/code can be accessed from within the containers at build
> time.
> > > Then I made the Docker compose.yml configurable through environment
> > > variables to allow for them to run in either the original manner --
> i.e.
> > > pulling artifacts from public repos -- or in a "local" mode, where
> > > artifacts are pulled from local builds.
> > >
> > > This process is a cleaner if the playground is a subfolder of the
> > > flink-statefun project rather than be its own repository
> > > (flink-statefun-playground), because then all the relative paths
> between
> > > the playground files and the build artifacts are fixed. So, I'd like to
> > > propose to move the playground files, modified as described above, to
> > > flink-statefun/playground and retire flink-statefun-playground. I can
> > > submit separate PR s those changes if everyone is on board.
> > >
> > > Also, should I plan to do the same upgrade to handle Flink 1.17.x? It
> > > should be easy to do, especially while the 1.16.x upgrade is fresh on
> my
> > > mind.
> > >
> > > Thanks.
> > >
> > >
> > > On Fri, Aug 11, 2023 at 6:40 PM Galen Warren <ga...@cvillewarrens.com>
> > > wrote:
> > >
> > >> I'm done with the code to make Statefun compatible with Flink 1.16,
> and
> > >> all the tests (including e2e succeed). The required changes were
> pretty
> > >> minimal.
> > >>
> > >> I'm running into a bit of a chicken/egg problem executing the tests in
> > >> flink-statefun-playground
> > >> <https://github.com/apache/flink-statefun-playground>, though. That
> > >> project seems to assume that all the various Statefun artifacts are
> > built
> > >> and deployed to the various public repositories already. I've looked
> > into
> > >> trying to redirect references to local artifacts; however, that's also
> > >> tricky since all the building occurs in Docker containers.
> > >>
> > >> Gordon, is there a trick to running the sample code in
> > >> flink-statefun-playground against yet-unreleased code that I'm
> missing?
> > >>
> > >> Thanks.
> > >>
> > >> On Sat, Jun 24, 2023 at 12:40 PM Galen Warren <
> galen@cvillewarrens.com>
> > >> wrote:
> > >>
> > >>> Great -- thanks!
> > >>>
> > >>> I'm going to be out of town for about a week but I'll take a look at
> > >>> this when I'm back.
> > >>>
> > >>> On Tue, Jun 20, 2023 at 8:46 AM Martijn Visser <mvisser@confluent.io
> >
> > >>> wrote:
> > >>>
> > >>>> Hi Galen,
> > >>>>
> > >>>> Yes, I'll be more than happy to help with Statefun releases.
> > >>>>
> > >>>> Best regards,
> > >>>>
> > >>>> Martijn
> > >>>>
> > >>>> On Tue, Jun 20, 2023 at 2:21 PM Galen Warren <
> galen@cvillewarrens.com
> > >
> > >>>> wrote:
> > >>>>
> > >>>>> Thanks.
> > >>>>>
> > >>>>> Martijn, to answer your question, I'd need to do a small amount of
> > >>>>> work to get a PR ready, but not much. Happy to do it if we're
> > deciding to
> > >>>>> restart Statefun releases -- are we?
> > >>>>>
> > >>>>> -- Galen
> > >>>>>
> > >>>>> On Sat, Jun 17, 2023 at 9:47 AM Tzu-Li (Gordon) Tai <
> > >>>>> tzulitai@apache.org> wrote:
> > >>>>>
> > >>>>>> > Perhaps he could weigh in on whether the combination of
> automated
> > >>>>>> tests plus those smoke tests should be sufficient for testing with
> > new
> > >>>>>> Flink versions
> > >>>>>>
> > >>>>>> What we usually did at the bare minimum for new StateFun releases
> > was
> > >>>>>> the following:
> > >>>>>>
> > >>>>>>    1. Build tests (including the smoke tests in the e2e module,
> > >>>>>>    which covers important tests like exactly-once verification)
> > >>>>>>    2. Updating the flink-statefun-playground repo and manually
> > >>>>>>    running all language examples there.
> > >>>>>>
> > >>>>>> If upgrading Flink versions was the only change in the release,
> I'd
> > >>>>>> probably say that this is sufficient.
> > >>>>>>
> > >>>>>> Best,
> > >>>>>> Gordon
> > >>>>>>
> > >>>>>> On Thu, Jun 15, 2023 at 5:25 AM Martijn Visser <
> > >>>>>> martijnvisser@apache.org> wrote:
> > >>>>>>
> > >>>>>>> Let me know if you have a PR for a Flink update :)
> > >>>>>>>
> > >>>>>>> On Thu, Jun 8, 2023 at 5:52 PM Galen Warren via user <
> > >>>>>>> user@flink.apache.org> wrote:
> > >>>>>>>
> > >>>>>>>> Thanks Martijn.
> > >>>>>>>>
> > >>>>>>>> Personally, I'm already using a local fork of Statefun that is
> > >>>>>>>> compatible with Flink 1.16.x, so I wouldn't have any need for a
> > released
> > >>>>>>>> version compatible with 1.15.x. I'd be happy to do the PRs to
> > modify
> > >>>>>>>> Statefun to work with new versions of Flink as they come along.
> > >>>>>>>>
> > >>>>>>>> As for testing, Statefun does have unit tests and Gordon also
> sent
> > >>>>>>>> me instructions a while back for how to do some additional smoke
> > tests
> > >>>>>>>> which are pretty straightforward. Perhaps he could weigh in on
> > whether the
> > >>>>>>>> combination of automated tests plus those smoke tests should be
> > sufficient
> > >>>>>>>> for testing with new Flink versions (I believe the answer is
> yes).
> > >>>>>>>>
> > >>>>>>>> -- Galen
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On Thu, Jun 8, 2023 at 8:01 AM Martijn Visser <
> > >>>>>>>> martijnvisser@apache.org> wrote:
> > >>>>>>>>
> > >>>>>>>>> Hi all,
> > >>>>>>>>>
> > >>>>>>>>> Apologies for the late reply.
> > >>>>>>>>>
> > >>>>>>>>> I'm willing to help out with merging requests in Statefun to
> keep
> > >>>>>>>>> them
> > >>>>>>>>> compatible with new Flink releases and create new releases. I
> do
> > >>>>>>>>> think that
> > >>>>>>>>> validation of the functionality of these releases depends a lot
> > on
> > >>>>>>>>> those
> > >>>>>>>>> who do these compatibility updates, with PMC members helping
> out
> > >>>>>>>>> with the
> > >>>>>>>>> formal process.
> > >>>>>>>>>
> > >>>>>>>>> > Why can't the Apache Software Foundation allow community
> > members
> > >>>>>>>>> to bring
> > >>>>>>>>> it up to date?
> > >>>>>>>>>
> > >>>>>>>>> There's nothing preventing anyone from reviewing any of the
> > >>>>>>>>> current PRs or
> > >>>>>>>>> opening new ones. However, none of them are approved [1], so
> > >>>>>>>>> there's also
> > >>>>>>>>> nothing to merge.
> > >>>>>>>>>
> > >>>>>>>>> > I believe that there are people and companies on this mailing
> > >>>>>>>>> list
> > >>>>>>>>> interested in supporting Apache Flink Stateful Functions.
> > >>>>>>>>>
> > >>>>>>>>> If so, then now is the time to show.
> > >>>>>>>>>
> > >>>>>>>>> Would there be a preference to create a release with Galen's
> > merged
> > >>>>>>>>> compatibility update to Flink 1.15.2, or do we want to skip
> that
> > >>>>>>>>> and go
> > >>>>>>>>> straight to a newer version?
> > >>>>>>>>>
> > >>>>>>>>> Best regards,
> > >>>>>>>>>
> > >>>>>>>>> Martijn
> > >>>>>>>>>
> > >>>>>>>>> [1]
> > >>>>>>>>>
> > >>>>>>>>>
> >
> https://github.com/apache/flink-statefun/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved
> > >>>>>>>>>
> > >>>>>>>>> On Tue, Jun 6, 2023 at 3:55 PM Marco Villalobos <
> > >>>>>>>>> mvillalobos@kineteque.com>
> > >>>>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>> > Why can't the Apache Software Foundation allow community
> > members
> > >>>>>>>>> to bring
> > >>>>>>>>> > it up to date?
> > >>>>>>>>> >
> > >>>>>>>>> > What's the process for that?
> > >>>>>>>>> >
> > >>>>>>>>> > I believe that there are people and companies on this mailing
> > >>>>>>>>> list
> > >>>>>>>>> > interested in supporting Apache Flink Stateful Functions.
> > >>>>>>>>> >
> > >>>>>>>>> > You already had two people on this thread express interest.
> > >>>>>>>>> >
> > >>>>>>>>> > At the very least, we could keep the library versions up to
> > date.
> > >>>>>>>>> >
> > >>>>>>>>> > There are only a small list of new features that might be
> > >>>>>>>>> worthwhile:
> > >>>>>>>>> >
> > >>>>>>>>> > 1. event time processing
> > >>>>>>>>> > 2. state rest api
> > >>>>>>>>> >
> > >>>>>>>>> >
> > >>>>>>>>> > On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <
> > chesnay@apache.org>
> > >>>>>>>>> wrote:
> > >>>>>>>>> >
> > >>>>>>>>> > If you were to fork it *and want to redistribute it* then the
> > >>>>>>>>> short
> > >>>>>>>>> > version is that
> > >>>>>>>>> >
> > >>>>>>>>> >    1. you have to adhere to the Apache licensing requirements
> > >>>>>>>>> >    2. you have to make it clear that your fork does not
> belong
> > >>>>>>>>> to the
> > >>>>>>>>> >    Apache Flink project. (Trademarks and all that)
> > >>>>>>>>> >
> > >>>>>>>>> > Neither should be significant hurdles (there should also be
> > >>>>>>>>> plenty of
> > >>>>>>>>> > online resources regarding 1), and if you do this then you
> can
> > >>>>>>>>> freely share
> > >>>>>>>>> > your fork with others.
> > >>>>>>>>> >
> > >>>>>>>>> > I've also pinged Martijn to take a look at this thread.
> > >>>>>>>>> > To my knowledge the project hasn't decided anything yet.
> > >>>>>>>>> >
> > >>>>>>>>> > On 27/05/2023 04:05, Galen Warren wrote:
> > >>>>>>>>> >
> > >>>>>>>>> > Ok, I get it. No interest.
> > >>>>>>>>> >
> > >>>>>>>>> > If this project is being abandoned, I guess I'll work with my
> > >>>>>>>>> own fork. Is
> > >>>>>>>>> > there anything I should consider here? Can I share it with
> > other
> > >>>>>>>>> people who
> > >>>>>>>>> > use this project?
> > >>>>>>>>> >
> > >>>>>>>>> > On Tue, May 16, 2023 at 10:50 AM Galen Warren <
> > >>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
> > >>>>>>>>> > wrote:
> > >>>>>>>>> >
> > >>>>>>>>> >
> > >>>>>>>>> > Hi Martijn, since you opened this discussion thread, I'm
> > curious
> > >>>>>>>>> what your
> > >>>>>>>>> > thoughts are in light of the responses? Thanks.
> > >>>>>>>>> >
> > >>>>>>>>> > On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <
> > >>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
> > >>>>>>>>> > wrote:
> > >>>>>>>>> >
> > >>>>>>>>> >
> > >>>>>>>>> > I use Apache Flink for stream processing, and StateFun as a
> > >>>>>>>>> hand-off
> > >>>>>>>>> >
> > >>>>>>>>> > point for the rest of the application.
> > >>>>>>>>> > It serves well as a bridge between a Flink Streaming job and
> > >>>>>>>>> > micro-services.
> > >>>>>>>>> >
> > >>>>>>>>> > This is essentially how I use it as well, and I would also be
> > >>>>>>>>> sad to see
> > >>>>>>>>> > it sunsetted. It works well; I don't know that there is a lot
> > of
> > >>>>>>>>> new
> > >>>>>>>>> > development required, but if there are no new Statefun
> > releases,
> > >>>>>>>>> then
> > >>>>>>>>> > Statefun can only be used with older Flink versions.
> > >>>>>>>>> >
> > >>>>>>>>> > On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
> > >>>>>>>>> mvillalobos@kineteque.com> wrote:
> > >>>>>>>>> >
> > >>>>>>>>> >
> > >>>>>>>>> > I am currently using Stateful Functions in my application.
> > >>>>>>>>> >
> > >>>>>>>>> > I use Apache Flink for stream processing, and StateFun as a
> > >>>>>>>>> hand-off
> > >>>>>>>>> > point for the rest of the application.
> > >>>>>>>>> > It serves well as a bridge between a Flink Streaming job and
> > >>>>>>>>> > micro-services.
> > >>>>>>>>> >
> > >>>>>>>>> > I would be disappointed if StateFun was sunsetted.  Its a
> good
> > >>>>>>>>> idea.
> > >>>>>>>>> >
> > >>>>>>>>> > If there is anything I can do to help, as a contributor
> > perhaps,
> > >>>>>>>>> please
> > >>>>>>>>> > let me know.
> > >>>>>>>>> >
> > >>>>>>>>> >
> > >>>>>>>>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <
> > >>>>>>>>> martijnvisser@apache.org> <ma...@apache.org>
> > >>>>>>>>> >
> > >>>>>>>>> > wrote:
> > >>>>>>>>> >
> > >>>>>>>>> > Hi everyone,
> > >>>>>>>>> >
> > >>>>>>>>> > I want to open a discussion on the status of the Statefun
> > >>>>>>>>> Project [1]
> > >>>>>>>>> >
> > >>>>>>>>> > in Apache Flink. As you might have noticed, there hasn't been
> > >>>>>>>>> much
> > >>>>>>>>> > development over the past months in the Statefun repository
> > [2].
> > >>>>>>>>> There is
> > >>>>>>>>> > currently a lack of active contributors and committers who
> are
> > >>>>>>>>> able to help
> > >>>>>>>>> > with the maintenance of the project.
> > >>>>>>>>> >
> > >>>>>>>>> > In order to improve the situation, we need to solve the lack
> of
> > >>>>>>>>> >
> > >>>>>>>>> > committers and the lack of contributors.
> > >>>>>>>>> >
> > >>>>>>>>> > On the lack of committers:
> > >>>>>>>>> >
> > >>>>>>>>> > 1. Ideally, there are some of the current Flink committers
> who
> > >>>>>>>>> have
> > >>>>>>>>> >
> > >>>>>>>>> > the bandwidth and can help with reviewing PRs and merging
> them.
> > >>>>>>>>> >
> > >>>>>>>>> > 2. If that's not an option, it could be a consideration that
> > >>>>>>>>> current
> > >>>>>>>>> >
> > >>>>>>>>> > committers only approve and review PRs, that are approved by
> > >>>>>>>>> those who are
> > >>>>>>>>> > willing to contribute to Statefun and if the CI passes
> > >>>>>>>>> >
> > >>>>>>>>> > On the lack of contributors:
> > >>>>>>>>> >
> > >>>>>>>>> > 3. Next to having this discussion on the Dev and User mailing
> > >>>>>>>>> list, we
> > >>>>>>>>> >
> > >>>>>>>>> > can also create a blog with a call for new contributors on
> the
> > >>>>>>>>> Flink
> > >>>>>>>>> > project website, send out some tweets on the Flink / Statefun
> > >>>>>>>>> twitter
> > >>>>>>>>> > accounts, post messages on Slack etc. In that message, we
> would
> > >>>>>>>>> inform how
> > >>>>>>>>> > those that are interested in contributing can start and where
> > >>>>>>>>> they could
> > >>>>>>>>> > reach out for more information.
> > >>>>>>>>> >
> > >>>>>>>>> > There's also option 4. where a group of interested people
> would
> > >>>>>>>>> split
> > >>>>>>>>> >
> > >>>>>>>>> > Statefun from the Flink project and make it a separate top
> > level
> > >>>>>>>>> project
> > >>>>>>>>> > under the Apache Flink umbrella (similar as recently has
> > >>>>>>>>> happened with
> > >>>>>>>>> > Flink Table Store, which has become Apache Paimon).
> > >>>>>>>>> >
> > >>>>>>>>> > If we see no improvements in the coming period, we should
> > >>>>>>>>> consider
> > >>>>>>>>> >
> > >>>>>>>>> > sunsetting Statefun and communicate that clearly to the
> users.
> > >>>>>>>>> >
> > >>>>>>>>> > I'm looking forward to your thoughts.
> > >>>>>>>>> >
> > >>>>>>>>> > Best regards,
> > >>>>>>>>> >
> > >>>>>>>>> > Martijn
> > >>>>>>>>> >
> > >>>>>>>>> > [1]
> > >>>>>>>>> https://nightlies.apache.org/flink/flink-statefun-docs-master/
> <
> > >>>>>>>>> >
> > >>>>>>>>> >
> https://nightlies.apache.org/flink/flink-statefun-docs-master/
> > >
> > >>>>>>>>> >
> > >>>>>>>>> > [2] https://github.com/apache/flink-statefun <
> > >>>>>>>>> >
> > >>>>>>>>> > https://github.com/apache/flink-statefun>
> > >>>>>>>>> >
> > >>>>>>>>> >
> > >>>>>>>>> >
> > >>>>>>>>> >
> > >>>>>>>>> >
> > >>>>>>>>>
> > >>>>>>>>
> >
>

Re: [DISCUSS] Status of Statefun Project

["Tzu-Li (Gordon) Tai" <tz...@apache.org>]

Hi Galen,

> locally built code is copied into the build containers
so that it can be accessed during the build.

That's exactly what we had been doing for release testing, yes. Sorry I
missed that detail in my previous response.

And yes, that sounds like a reasonable approach. If I understand you
correctly, the workflow would become this:

   1. Build the StateFun repo locally to install the snapshot artifact
   jars + have a local base StateFun image.
   2. Run the playground in "local" mode, so that it uses the local base
   StateFun image + builds the playground code using copied artifact jars
   (instead of pulling from Maven).

That looks good to me!

Thanks,
Gordon

On Fri, Aug 18, 2023 at 11:33 AM Galen Warren
<ga...@cvillewarrens.com.invalid> wrote:

> Thanks.
>
> If you were to build a local image, as you suggest, how do you access that
> image when building the playground images? All the compilation of
> playground code happens inside containers, so local images on the host
> aren't available in those containers. Unless I'm missing something?
>
> I've slightly reworked things such that the playground images can be run in
> one of two modes -- the default mode, which works like before, and a
> "local" mode where locally built code is copied into the build containers
> so that it can be accessed during the build. It works fine, you just have
> to define a couple of environment variables when running docker-compose to
> specify default vs. local mode and what versions of Flink and Statefun
> should be referenced, and then you can build a run the local examples
> without any additional steps. Does that sound like a reasonable approach?
>
>
> On Fri, Aug 18, 2023 at 2:17 PM Tzu-Li (Gordon) Tai <tz...@apache.org>
> wrote:
>
> > Hi Galen,
> >
> > > Gordon, is there a trick to running the sample code in
> > flink-statefun-playground against yet-unreleased code that I'm missing?
> >
> > You'd have to locally build an image from the release branch, with a
> > temporary image version tag. Then, in the flink-statefun-playground,
> change
> > the image versions in the docker-compose files to use that locally built
> > image. IIRC, that's what we have been doing in the past. Admittedly, it's
> > pretty manual - I don't think the CI manages this workflow.
> >
> > Thanks,
> > Gordon
> >
> > On Mon, Aug 14, 2023 at 10:42 AM Galen Warren <ga...@cvillewarrens.com>
> > wrote:
> >
> > > I created a pull request for this: [FLINK-31619] Upgrade Stateful
> > > Functions to Flink 1.16.1 by galenwarren · Pull Request #331 ·
> > > apache/flink-statefun (github.com)
> > > <https://github.com/apache/flink-statefun/pull/331>.
> > >
> > > JIRA is here: [FLINK-31619] Upgrade Stateful Functions to Flink 1.16.1
> -
> > > ASF JIRA (apache.org)
> > > <https://issues.apache.org/jira/browse/FLINK-31619?filter=-1>.
> > >
> > > Statefun references 1.16.2, despite the title -- that version has come
> > out
> > > since the issue was created.
> > >
> > > I figured out how to run all the playground tests locally, but it took
> a
> > > bit of reworking of the playground setup with respect to Docker;
> > > specifically, the Docker contexts used to build the example functions
> > > needed to be broadened (i.e. moved up the tree) so that, if needed,
> local
> > > artifacts/code can be accessed from within the containers at build
> time.
> > > Then I made the Docker compose.yml configurable through environment
> > > variables to allow for them to run in either the original manner --
> i.e.
> > > pulling artifacts from public repos -- or in a "local" mode, where
> > > artifacts are pulled from local builds.
> > >
> > > This process is a cleaner if the playground is a subfolder of the
> > > flink-statefun project rather than be its own repository
> > > (flink-statefun-playground), because then all the relative paths
> between
> > > the playground files and the build artifacts are fixed. So, I'd like to
> > > propose to move the playground files, modified as described above, to
> > > flink-statefun/playground and retire flink-statefun-playground. I can
> > > submit separate PR s those changes if everyone is on board.
> > >
> > > Also, should I plan to do the same upgrade to handle Flink 1.17.x? It
> > > should be easy to do, especially while the 1.16.x upgrade is fresh on
> my
> > > mind.
> > >
> > > Thanks.
> > >
> > >
> > > On Fri, Aug 11, 2023 at 6:40 PM Galen Warren <ga...@cvillewarrens.com>
> > > wrote:
> > >
> > >> I'm done with the code to make Statefun compatible with Flink 1.16,
> and
> > >> all the tests (including e2e succeed). The required changes were
> pretty
> > >> minimal.
> > >>
> > >> I'm running into a bit of a chicken/egg problem executing the tests in
> > >> flink-statefun-playground
> > >> <https://github.com/apache/flink-statefun-playground>, though. That
> > >> project seems to assume that all the various Statefun artifacts are
> > built
> > >> and deployed to the various public repositories already. I've looked
> > into
> > >> trying to redirect references to local artifacts; however, that's also
> > >> tricky since all the building occurs in Docker containers.
> > >>
> > >> Gordon, is there a trick to running the sample code in
> > >> flink-statefun-playground against yet-unreleased code that I'm
> missing?
> > >>
> > >> Thanks.
> > >>
> > >> On Sat, Jun 24, 2023 at 12:40 PM Galen Warren <
> galen@cvillewarrens.com>
> > >> wrote:
> > >>
> > >>> Great -- thanks!
> > >>>
> > >>> I'm going to be out of town for about a week but I'll take a look at
> > >>> this when I'm back.
> > >>>
> > >>> On Tue, Jun 20, 2023 at 8:46 AM Martijn Visser <mvisser@confluent.io
> >
> > >>> wrote:
> > >>>
> > >>>> Hi Galen,
> > >>>>
> > >>>> Yes, I'll be more than happy to help with Statefun releases.
> > >>>>
> > >>>> Best regards,
> > >>>>
> > >>>> Martijn
> > >>>>
> > >>>> On Tue, Jun 20, 2023 at 2:21 PM Galen Warren <
> galen@cvillewarrens.com
> > >
> > >>>> wrote:
> > >>>>
> > >>>>> Thanks.
> > >>>>>
> > >>>>> Martijn, to answer your question, I'd need to do a small amount of
> > >>>>> work to get a PR ready, but not much. Happy to do it if we're
> > deciding to
> > >>>>> restart Statefun releases -- are we?
> > >>>>>
> > >>>>> -- Galen
> > >>>>>
> > >>>>> On Sat, Jun 17, 2023 at 9:47 AM Tzu-Li (Gordon) Tai <
> > >>>>> tzulitai@apache.org> wrote:
> > >>>>>
> > >>>>>> > Perhaps he could weigh in on whether the combination of
> automated
> > >>>>>> tests plus those smoke tests should be sufficient for testing with
> > new
> > >>>>>> Flink versions
> > >>>>>>
> > >>>>>> What we usually did at the bare minimum for new StateFun releases
> > was
> > >>>>>> the following:
> > >>>>>>
> > >>>>>>    1. Build tests (including the smoke tests in the e2e module,
> > >>>>>>    which covers important tests like exactly-once verification)
> > >>>>>>    2. Updating the flink-statefun-playground repo and manually
> > >>>>>>    running all language examples there.
> > >>>>>>
> > >>>>>> If upgrading Flink versions was the only change in the release,
> I'd
> > >>>>>> probably say that this is sufficient.
> > >>>>>>
> > >>>>>> Best,
> > >>>>>> Gordon
> > >>>>>>
> > >>>>>> On Thu, Jun 15, 2023 at 5:25 AM Martijn Visser <
> > >>>>>> martijnvisser@apache.org> wrote:
> > >>>>>>
> > >>>>>>> Let me know if you have a PR for a Flink update :)
> > >>>>>>>
> > >>>>>>> On Thu, Jun 8, 2023 at 5:52 PM Galen Warren via user <
> > >>>>>>> user@flink.apache.org> wrote:
> > >>>>>>>
> > >>>>>>>> Thanks Martijn.
> > >>>>>>>>
> > >>>>>>>> Personally, I'm already using a local fork of Statefun that is
> > >>>>>>>> compatible with Flink 1.16.x, so I wouldn't have any need for a
> > released
> > >>>>>>>> version compatible with 1.15.x. I'd be happy to do the PRs to
> > modify
> > >>>>>>>> Statefun to work with new versions of Flink as they come along.
> > >>>>>>>>
> > >>>>>>>> As for testing, Statefun does have unit tests and Gordon also
> sent
> > >>>>>>>> me instructions a while back for how to do some additional smoke
> > tests
> > >>>>>>>> which are pretty straightforward. Perhaps he could weigh in on
> > whether the
> > >>>>>>>> combination of automated tests plus those smoke tests should be
> > sufficient
> > >>>>>>>> for testing with new Flink versions (I believe the answer is
> yes).
> > >>>>>>>>
> > >>>>>>>> -- Galen
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On Thu, Jun 8, 2023 at 8:01 AM Martijn Visser <
> > >>>>>>>> martijnvisser@apache.org> wrote:
> > >>>>>>>>
> > >>>>>>>>> Hi all,
> > >>>>>>>>>
> > >>>>>>>>> Apologies for the late reply.
> > >>>>>>>>>
> > >>>>>>>>> I'm willing to help out with merging requests in Statefun to
> keep
> > >>>>>>>>> them
> > >>>>>>>>> compatible with new Flink releases and create new releases. I
> do
> > >>>>>>>>> think that
> > >>>>>>>>> validation of the functionality of these releases depends a lot
> > on
> > >>>>>>>>> those
> > >>>>>>>>> who do these compatibility updates, with PMC members helping
> out
> > >>>>>>>>> with the
> > >>>>>>>>> formal process.
> > >>>>>>>>>
> > >>>>>>>>> > Why can't the Apache Software Foundation allow community
> > members
> > >>>>>>>>> to bring
> > >>>>>>>>> it up to date?
> > >>>>>>>>>
> > >>>>>>>>> There's nothing preventing anyone from reviewing any of the
> > >>>>>>>>> current PRs or
> > >>>>>>>>> opening new ones. However, none of them are approved [1], so
> > >>>>>>>>> there's also
> > >>>>>>>>> nothing to merge.
> > >>>>>>>>>
> > >>>>>>>>> > I believe that there are people and companies on this mailing
> > >>>>>>>>> list
> > >>>>>>>>> interested in supporting Apache Flink Stateful Functions.
> > >>>>>>>>>
> > >>>>>>>>> If so, then now is the time to show.
> > >>>>>>>>>
> > >>>>>>>>> Would there be a preference to create a release with Galen's
> > merged
> > >>>>>>>>> compatibility update to Flink 1.15.2, or do we want to skip
> that
> > >>>>>>>>> and go
> > >>>>>>>>> straight to a newer version?
> > >>>>>>>>>
> > >>>>>>>>> Best regards,
> > >>>>>>>>>
> > >>>>>>>>> Martijn
> > >>>>>>>>>
> > >>>>>>>>> [1]
> > >>>>>>>>>
> > >>>>>>>>>
> >
> https://github.com/apache/flink-statefun/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved
> > >>>>>>>>>
> > >>>>>>>>> On Tue, Jun 6, 2023 at 3:55 PM Marco Villalobos <
> > >>>>>>>>> mvillalobos@kineteque.com>
> > >>>>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>> > Why can't the Apache Software Foundation allow community
> > members
> > >>>>>>>>> to bring
> > >>>>>>>>> > it up to date?
> > >>>>>>>>> >
> > >>>>>>>>> > What's the process for that?
> > >>>>>>>>> >
> > >>>>>>>>> > I believe that there are people and companies on this mailing
> > >>>>>>>>> list
> > >>>>>>>>> > interested in supporting Apache Flink Stateful Functions.
> > >>>>>>>>> >
> > >>>>>>>>> > You already had two people on this thread express interest.
> > >>>>>>>>> >
> > >>>>>>>>> > At the very least, we could keep the library versions up to
> > date.
> > >>>>>>>>> >
> > >>>>>>>>> > There are only a small list of new features that might be
> > >>>>>>>>> worthwhile:
> > >>>>>>>>> >
> > >>>>>>>>> > 1. event time processing
> > >>>>>>>>> > 2. state rest api
> > >>>>>>>>> >
> > >>>>>>>>> >
> > >>>>>>>>> > On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <
> > chesnay@apache.org>
> > >>>>>>>>> wrote:
> > >>>>>>>>> >
> > >>>>>>>>> > If you were to fork it *and want to redistribute it* then the
> > >>>>>>>>> short
> > >>>>>>>>> > version is that
> > >>>>>>>>> >
> > >>>>>>>>> >    1. you have to adhere to the Apache licensing requirements
> > >>>>>>>>> >    2. you have to make it clear that your fork does not
> belong
> > >>>>>>>>> to the
> > >>>>>>>>> >    Apache Flink project. (Trademarks and all that)
> > >>>>>>>>> >
> > >>>>>>>>> > Neither should be significant hurdles (there should also be
> > >>>>>>>>> plenty of
> > >>>>>>>>> > online resources regarding 1), and if you do this then you
> can
> > >>>>>>>>> freely share
> > >>>>>>>>> > your fork with others.
> > >>>>>>>>> >
> > >>>>>>>>> > I've also pinged Martijn to take a look at this thread.
> > >>>>>>>>> > To my knowledge the project hasn't decided anything yet.
> > >>>>>>>>> >
> > >>>>>>>>> > On 27/05/2023 04:05, Galen Warren wrote:
> > >>>>>>>>> >
> > >>>>>>>>> > Ok, I get it. No interest.
> > >>>>>>>>> >
> > >>>>>>>>> > If this project is being abandoned, I guess I'll work with my
> > >>>>>>>>> own fork. Is
> > >>>>>>>>> > there anything I should consider here? Can I share it with
> > other
> > >>>>>>>>> people who
> > >>>>>>>>> > use this project?
> > >>>>>>>>> >
> > >>>>>>>>> > On Tue, May 16, 2023 at 10:50 AM Galen Warren <
> > >>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
> > >>>>>>>>> > wrote:
> > >>>>>>>>> >
> > >>>>>>>>> >
> > >>>>>>>>> > Hi Martijn, since you opened this discussion thread, I'm
> > curious
> > >>>>>>>>> what your
> > >>>>>>>>> > thoughts are in light of the responses? Thanks.
> > >>>>>>>>> >
> > >>>>>>>>> > On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <
> > >>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
> > >>>>>>>>> > wrote:
> > >>>>>>>>> >
> > >>>>>>>>> >
> > >>>>>>>>> > I use Apache Flink for stream processing, and StateFun as a
> > >>>>>>>>> hand-off
> > >>>>>>>>> >
> > >>>>>>>>> > point for the rest of the application.
> > >>>>>>>>> > It serves well as a bridge between a Flink Streaming job and
> > >>>>>>>>> > micro-services.
> > >>>>>>>>> >
> > >>>>>>>>> > This is essentially how I use it as well, and I would also be
> > >>>>>>>>> sad to see
> > >>>>>>>>> > it sunsetted. It works well; I don't know that there is a lot
> > of
> > >>>>>>>>> new
> > >>>>>>>>> > development required, but if there are no new Statefun
> > releases,
> > >>>>>>>>> then
> > >>>>>>>>> > Statefun can only be used with older Flink versions.
> > >>>>>>>>> >
> > >>>>>>>>> > On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
> > >>>>>>>>> mvillalobos@kineteque.com> wrote:
> > >>>>>>>>> >
> > >>>>>>>>> >
> > >>>>>>>>> > I am currently using Stateful Functions in my application.
> > >>>>>>>>> >
> > >>>>>>>>> > I use Apache Flink for stream processing, and StateFun as a
> > >>>>>>>>> hand-off
> > >>>>>>>>> > point for the rest of the application.
> > >>>>>>>>> > It serves well as a bridge between a Flink Streaming job and
> > >>>>>>>>> > micro-services.
> > >>>>>>>>> >
> > >>>>>>>>> > I would be disappointed if StateFun was sunsetted.  Its a
> good
> > >>>>>>>>> idea.
> > >>>>>>>>> >
> > >>>>>>>>> > If there is anything I can do to help, as a contributor
> > perhaps,
> > >>>>>>>>> please
> > >>>>>>>>> > let me know.
> > >>>>>>>>> >
> > >>>>>>>>> >
> > >>>>>>>>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <
> > >>>>>>>>> martijnvisser@apache.org> <ma...@apache.org>
> > >>>>>>>>> >
> > >>>>>>>>> > wrote:
> > >>>>>>>>> >
> > >>>>>>>>> > Hi everyone,
> > >>>>>>>>> >
> > >>>>>>>>> > I want to open a discussion on the status of the Statefun
> > >>>>>>>>> Project [1]
> > >>>>>>>>> >
> > >>>>>>>>> > in Apache Flink. As you might have noticed, there hasn't been
> > >>>>>>>>> much
> > >>>>>>>>> > development over the past months in the Statefun repository
> > [2].
> > >>>>>>>>> There is
> > >>>>>>>>> > currently a lack of active contributors and committers who
> are
> > >>>>>>>>> able to help
> > >>>>>>>>> > with the maintenance of the project.
> > >>>>>>>>> >
> > >>>>>>>>> > In order to improve the situation, we need to solve the lack
> of
> > >>>>>>>>> >
> > >>>>>>>>> > committers and the lack of contributors.
> > >>>>>>>>> >
> > >>>>>>>>> > On the lack of committers:
> > >>>>>>>>> >
> > >>>>>>>>> > 1. Ideally, there are some of the current Flink committers
> who
> > >>>>>>>>> have
> > >>>>>>>>> >
> > >>>>>>>>> > the bandwidth and can help with reviewing PRs and merging
> them.
> > >>>>>>>>> >
> > >>>>>>>>> > 2. If that's not an option, it could be a consideration that
> > >>>>>>>>> current
> > >>>>>>>>> >
> > >>>>>>>>> > committers only approve and review PRs, that are approved by
> > >>>>>>>>> those who are
> > >>>>>>>>> > willing to contribute to Statefun and if the CI passes
> > >>>>>>>>> >
> > >>>>>>>>> > On the lack of contributors:
> > >>>>>>>>> >
> > >>>>>>>>> > 3. Next to having this discussion on the Dev and User mailing
> > >>>>>>>>> list, we
> > >>>>>>>>> >
> > >>>>>>>>> > can also create a blog with a call for new contributors on
> the
> > >>>>>>>>> Flink
> > >>>>>>>>> > project website, send out some tweets on the Flink / Statefun
> > >>>>>>>>> twitter
> > >>>>>>>>> > accounts, post messages on Slack etc. In that message, we
> would
> > >>>>>>>>> inform how
> > >>>>>>>>> > those that are interested in contributing can start and where
> > >>>>>>>>> they could
> > >>>>>>>>> > reach out for more information.
> > >>>>>>>>> >
> > >>>>>>>>> > There's also option 4. where a group of interested people
> would
> > >>>>>>>>> split
> > >>>>>>>>> >
> > >>>>>>>>> > Statefun from the Flink project and make it a separate top
> > level
> > >>>>>>>>> project
> > >>>>>>>>> > under the Apache Flink umbrella (similar as recently has
> > >>>>>>>>> happened with
> > >>>>>>>>> > Flink Table Store, which has become Apache Paimon).
> > >>>>>>>>> >
> > >>>>>>>>> > If we see no improvements in the coming period, we should
> > >>>>>>>>> consider
> > >>>>>>>>> >
> > >>>>>>>>> > sunsetting Statefun and communicate that clearly to the
> users.
> > >>>>>>>>> >
> > >>>>>>>>> > I'm looking forward to your thoughts.
> > >>>>>>>>> >
> > >>>>>>>>> > Best regards,
> > >>>>>>>>> >
> > >>>>>>>>> > Martijn
> > >>>>>>>>> >
> > >>>>>>>>> > [1]
> > >>>>>>>>> https://nightlies.apache.org/flink/flink-statefun-docs-master/
> <
> > >>>>>>>>> >
> > >>>>>>>>> >
> https://nightlies.apache.org/flink/flink-statefun-docs-master/
> > >
> > >>>>>>>>> >
> > >>>>>>>>> > [2] https://github.com/apache/flink-statefun <
> > >>>>>>>>> >
> > >>>>>>>>> > https://github.com/apache/flink-statefun>
> > >>>>>>>>> >
> > >>>>>>>>> >
> > >>>>>>>>> >
> > >>>>>>>>> >
> > >>>>>>>>> >
> > >>>>>>>>>
> > >>>>>>>>
> >
>

Re: [DISCUSS] Status of Statefun Project

[Galen Warren via user <us...@flink.apache.org>]

Thanks.

If you were to build a local image, as you suggest, how do you access that
image when building the playground images? All the compilation of
playground code happens inside containers, so local images on the host
aren't available in those containers. Unless I'm missing something?

I've slightly reworked things such that the playground images can be run in
one of two modes -- the default mode, which works like before, and a
"local" mode where locally built code is copied into the build containers
so that it can be accessed during the build. It works fine, you just have
to define a couple of environment variables when running docker-compose to
specify default vs. local mode and what versions of Flink and Statefun
should be referenced, and then you can build a run the local examples
without any additional steps. Does that sound like a reasonable approach?


On Fri, Aug 18, 2023 at 2:17 PM Tzu-Li (Gordon) Tai <tz...@apache.org>
wrote:

> Hi Galen,
>
> > Gordon, is there a trick to running the sample code in
> flink-statefun-playground against yet-unreleased code that I'm missing?
>
> You'd have to locally build an image from the release branch, with a
> temporary image version tag. Then, in the flink-statefun-playground, change
> the image versions in the docker-compose files to use that locally built
> image. IIRC, that's what we have been doing in the past. Admittedly, it's
> pretty manual - I don't think the CI manages this workflow.
>
> Thanks,
> Gordon
>
> On Mon, Aug 14, 2023 at 10:42 AM Galen Warren <ga...@cvillewarrens.com>
> wrote:
>
> > I created a pull request for this: [FLINK-31619] Upgrade Stateful
> > Functions to Flink 1.16.1 by galenwarren · Pull Request #331 ·
> > apache/flink-statefun (github.com)
> > <https://github.com/apache/flink-statefun/pull/331>.
> >
> > JIRA is here: [FLINK-31619] Upgrade Stateful Functions to Flink 1.16.1 -
> > ASF JIRA (apache.org)
> > <https://issues.apache.org/jira/browse/FLINK-31619?filter=-1>.
> >
> > Statefun references 1.16.2, despite the title -- that version has come
> out
> > since the issue was created.
> >
> > I figured out how to run all the playground tests locally, but it took a
> > bit of reworking of the playground setup with respect to Docker;
> > specifically, the Docker contexts used to build the example functions
> > needed to be broadened (i.e. moved up the tree) so that, if needed, local
> > artifacts/code can be accessed from within the containers at build time.
> > Then I made the Docker compose.yml configurable through environment
> > variables to allow for them to run in either the original manner -- i.e.
> > pulling artifacts from public repos -- or in a "local" mode, where
> > artifacts are pulled from local builds.
> >
> > This process is a cleaner if the playground is a subfolder of the
> > flink-statefun project rather than be its own repository
> > (flink-statefun-playground), because then all the relative paths between
> > the playground files and the build artifacts are fixed. So, I'd like to
> > propose to move the playground files, modified as described above, to
> > flink-statefun/playground and retire flink-statefun-playground. I can
> > submit separate PR s those changes if everyone is on board.
> >
> > Also, should I plan to do the same upgrade to handle Flink 1.17.x? It
> > should be easy to do, especially while the 1.16.x upgrade is fresh on my
> > mind.
> >
> > Thanks.
> >
> >
> > On Fri, Aug 11, 2023 at 6:40 PM Galen Warren <ga...@cvillewarrens.com>
> > wrote:
> >
> >> I'm done with the code to make Statefun compatible with Flink 1.16, and
> >> all the tests (including e2e succeed). The required changes were pretty
> >> minimal.
> >>
> >> I'm running into a bit of a chicken/egg problem executing the tests in
> >> flink-statefun-playground
> >> <https://github.com/apache/flink-statefun-playground>, though. That
> >> project seems to assume that all the various Statefun artifacts are
> built
> >> and deployed to the various public repositories already. I've looked
> into
> >> trying to redirect references to local artifacts; however, that's also
> >> tricky since all the building occurs in Docker containers.
> >>
> >> Gordon, is there a trick to running the sample code in
> >> flink-statefun-playground against yet-unreleased code that I'm missing?
> >>
> >> Thanks.
> >>
> >> On Sat, Jun 24, 2023 at 12:40 PM Galen Warren <ga...@cvillewarrens.com>
> >> wrote:
> >>
> >>> Great -- thanks!
> >>>
> >>> I'm going to be out of town for about a week but I'll take a look at
> >>> this when I'm back.
> >>>
> >>> On Tue, Jun 20, 2023 at 8:46 AM Martijn Visser <mv...@confluent.io>
> >>> wrote:
> >>>
> >>>> Hi Galen,
> >>>>
> >>>> Yes, I'll be more than happy to help with Statefun releases.
> >>>>
> >>>> Best regards,
> >>>>
> >>>> Martijn
> >>>>
> >>>> On Tue, Jun 20, 2023 at 2:21 PM Galen Warren <galen@cvillewarrens.com
> >
> >>>> wrote:
> >>>>
> >>>>> Thanks.
> >>>>>
> >>>>> Martijn, to answer your question, I'd need to do a small amount of
> >>>>> work to get a PR ready, but not much. Happy to do it if we're
> deciding to
> >>>>> restart Statefun releases -- are we?
> >>>>>
> >>>>> -- Galen
> >>>>>
> >>>>> On Sat, Jun 17, 2023 at 9:47 AM Tzu-Li (Gordon) Tai <
> >>>>> tzulitai@apache.org> wrote:
> >>>>>
> >>>>>> > Perhaps he could weigh in on whether the combination of automated
> >>>>>> tests plus those smoke tests should be sufficient for testing with
> new
> >>>>>> Flink versions
> >>>>>>
> >>>>>> What we usually did at the bare minimum for new StateFun releases
> was
> >>>>>> the following:
> >>>>>>
> >>>>>>    1. Build tests (including the smoke tests in the e2e module,
> >>>>>>    which covers important tests like exactly-once verification)
> >>>>>>    2. Updating the flink-statefun-playground repo and manually
> >>>>>>    running all language examples there.
> >>>>>>
> >>>>>> If upgrading Flink versions was the only change in the release, I'd
> >>>>>> probably say that this is sufficient.
> >>>>>>
> >>>>>> Best,
> >>>>>> Gordon
> >>>>>>
> >>>>>> On Thu, Jun 15, 2023 at 5:25 AM Martijn Visser <
> >>>>>> martijnvisser@apache.org> wrote:
> >>>>>>
> >>>>>>> Let me know if you have a PR for a Flink update :)
> >>>>>>>
> >>>>>>> On Thu, Jun 8, 2023 at 5:52 PM Galen Warren via user <
> >>>>>>> user@flink.apache.org> wrote:
> >>>>>>>
> >>>>>>>> Thanks Martijn.
> >>>>>>>>
> >>>>>>>> Personally, I'm already using a local fork of Statefun that is
> >>>>>>>> compatible with Flink 1.16.x, so I wouldn't have any need for a
> released
> >>>>>>>> version compatible with 1.15.x. I'd be happy to do the PRs to
> modify
> >>>>>>>> Statefun to work with new versions of Flink as they come along.
> >>>>>>>>
> >>>>>>>> As for testing, Statefun does have unit tests and Gordon also sent
> >>>>>>>> me instructions a while back for how to do some additional smoke
> tests
> >>>>>>>> which are pretty straightforward. Perhaps he could weigh in on
> whether the
> >>>>>>>> combination of automated tests plus those smoke tests should be
> sufficient
> >>>>>>>> for testing with new Flink versions (I believe the answer is yes).
> >>>>>>>>
> >>>>>>>> -- Galen
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Thu, Jun 8, 2023 at 8:01 AM Martijn Visser <
> >>>>>>>> martijnvisser@apache.org> wrote:
> >>>>>>>>
> >>>>>>>>> Hi all,
> >>>>>>>>>
> >>>>>>>>> Apologies for the late reply.
> >>>>>>>>>
> >>>>>>>>> I'm willing to help out with merging requests in Statefun to keep
> >>>>>>>>> them
> >>>>>>>>> compatible with new Flink releases and create new releases. I do
> >>>>>>>>> think that
> >>>>>>>>> validation of the functionality of these releases depends a lot
> on
> >>>>>>>>> those
> >>>>>>>>> who do these compatibility updates, with PMC members helping out
> >>>>>>>>> with the
> >>>>>>>>> formal process.
> >>>>>>>>>
> >>>>>>>>> > Why can't the Apache Software Foundation allow community
> members
> >>>>>>>>> to bring
> >>>>>>>>> it up to date?
> >>>>>>>>>
> >>>>>>>>> There's nothing preventing anyone from reviewing any of the
> >>>>>>>>> current PRs or
> >>>>>>>>> opening new ones. However, none of them are approved [1], so
> >>>>>>>>> there's also
> >>>>>>>>> nothing to merge.
> >>>>>>>>>
> >>>>>>>>> > I believe that there are people and companies on this mailing
> >>>>>>>>> list
> >>>>>>>>> interested in supporting Apache Flink Stateful Functions.
> >>>>>>>>>
> >>>>>>>>> If so, then now is the time to show.
> >>>>>>>>>
> >>>>>>>>> Would there be a preference to create a release with Galen's
> merged
> >>>>>>>>> compatibility update to Flink 1.15.2, or do we want to skip that
> >>>>>>>>> and go
> >>>>>>>>> straight to a newer version?
> >>>>>>>>>
> >>>>>>>>> Best regards,
> >>>>>>>>>
> >>>>>>>>> Martijn
> >>>>>>>>>
> >>>>>>>>> [1]
> >>>>>>>>>
> >>>>>>>>>
> https://github.com/apache/flink-statefun/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved
> >>>>>>>>>
> >>>>>>>>> On Tue, Jun 6, 2023 at 3:55 PM Marco Villalobos <
> >>>>>>>>> mvillalobos@kineteque.com>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>> > Why can't the Apache Software Foundation allow community
> members
> >>>>>>>>> to bring
> >>>>>>>>> > it up to date?
> >>>>>>>>> >
> >>>>>>>>> > What's the process for that?
> >>>>>>>>> >
> >>>>>>>>> > I believe that there are people and companies on this mailing
> >>>>>>>>> list
> >>>>>>>>> > interested in supporting Apache Flink Stateful Functions.
> >>>>>>>>> >
> >>>>>>>>> > You already had two people on this thread express interest.
> >>>>>>>>> >
> >>>>>>>>> > At the very least, we could keep the library versions up to
> date.
> >>>>>>>>> >
> >>>>>>>>> > There are only a small list of new features that might be
> >>>>>>>>> worthwhile:
> >>>>>>>>> >
> >>>>>>>>> > 1. event time processing
> >>>>>>>>> > 2. state rest api
> >>>>>>>>> >
> >>>>>>>>> >
> >>>>>>>>> > On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <
> chesnay@apache.org>
> >>>>>>>>> wrote:
> >>>>>>>>> >
> >>>>>>>>> > If you were to fork it *and want to redistribute it* then the
> >>>>>>>>> short
> >>>>>>>>> > version is that
> >>>>>>>>> >
> >>>>>>>>> >    1. you have to adhere to the Apache licensing requirements
> >>>>>>>>> >    2. you have to make it clear that your fork does not belong
> >>>>>>>>> to the
> >>>>>>>>> >    Apache Flink project. (Trademarks and all that)
> >>>>>>>>> >
> >>>>>>>>> > Neither should be significant hurdles (there should also be
> >>>>>>>>> plenty of
> >>>>>>>>> > online resources regarding 1), and if you do this then you can
> >>>>>>>>> freely share
> >>>>>>>>> > your fork with others.
> >>>>>>>>> >
> >>>>>>>>> > I've also pinged Martijn to take a look at this thread.
> >>>>>>>>> > To my knowledge the project hasn't decided anything yet.
> >>>>>>>>> >
> >>>>>>>>> > On 27/05/2023 04:05, Galen Warren wrote:
> >>>>>>>>> >
> >>>>>>>>> > Ok, I get it. No interest.
> >>>>>>>>> >
> >>>>>>>>> > If this project is being abandoned, I guess I'll work with my
> >>>>>>>>> own fork. Is
> >>>>>>>>> > there anything I should consider here? Can I share it with
> other
> >>>>>>>>> people who
> >>>>>>>>> > use this project?
> >>>>>>>>> >
> >>>>>>>>> > On Tue, May 16, 2023 at 10:50 AM Galen Warren <
> >>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
> >>>>>>>>> > wrote:
> >>>>>>>>> >
> >>>>>>>>> >
> >>>>>>>>> > Hi Martijn, since you opened this discussion thread, I'm
> curious
> >>>>>>>>> what your
> >>>>>>>>> > thoughts are in light of the responses? Thanks.
> >>>>>>>>> >
> >>>>>>>>> > On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <
> >>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
> >>>>>>>>> > wrote:
> >>>>>>>>> >
> >>>>>>>>> >
> >>>>>>>>> > I use Apache Flink for stream processing, and StateFun as a
> >>>>>>>>> hand-off
> >>>>>>>>> >
> >>>>>>>>> > point for the rest of the application.
> >>>>>>>>> > It serves well as a bridge between a Flink Streaming job and
> >>>>>>>>> > micro-services.
> >>>>>>>>> >
> >>>>>>>>> > This is essentially how I use it as well, and I would also be
> >>>>>>>>> sad to see
> >>>>>>>>> > it sunsetted. It works well; I don't know that there is a lot
> of
> >>>>>>>>> new
> >>>>>>>>> > development required, but if there are no new Statefun
> releases,
> >>>>>>>>> then
> >>>>>>>>> > Statefun can only be used with older Flink versions.
> >>>>>>>>> >
> >>>>>>>>> > On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
> >>>>>>>>> mvillalobos@kineteque.com> wrote:
> >>>>>>>>> >
> >>>>>>>>> >
> >>>>>>>>> > I am currently using Stateful Functions in my application.
> >>>>>>>>> >
> >>>>>>>>> > I use Apache Flink for stream processing, and StateFun as a
> >>>>>>>>> hand-off
> >>>>>>>>> > point for the rest of the application.
> >>>>>>>>> > It serves well as a bridge between a Flink Streaming job and
> >>>>>>>>> > micro-services.
> >>>>>>>>> >
> >>>>>>>>> > I would be disappointed if StateFun was sunsetted.  Its a good
> >>>>>>>>> idea.
> >>>>>>>>> >
> >>>>>>>>> > If there is anything I can do to help, as a contributor
> perhaps,
> >>>>>>>>> please
> >>>>>>>>> > let me know.
> >>>>>>>>> >
> >>>>>>>>> >
> >>>>>>>>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <
> >>>>>>>>> martijnvisser@apache.org> <ma...@apache.org>
> >>>>>>>>> >
> >>>>>>>>> > wrote:
> >>>>>>>>> >
> >>>>>>>>> > Hi everyone,
> >>>>>>>>> >
> >>>>>>>>> > I want to open a discussion on the status of the Statefun
> >>>>>>>>> Project [1]
> >>>>>>>>> >
> >>>>>>>>> > in Apache Flink. As you might have noticed, there hasn't been
> >>>>>>>>> much
> >>>>>>>>> > development over the past months in the Statefun repository
> [2].
> >>>>>>>>> There is
> >>>>>>>>> > currently a lack of active contributors and committers who are
> >>>>>>>>> able to help
> >>>>>>>>> > with the maintenance of the project.
> >>>>>>>>> >
> >>>>>>>>> > In order to improve the situation, we need to solve the lack of
> >>>>>>>>> >
> >>>>>>>>> > committers and the lack of contributors.
> >>>>>>>>> >
> >>>>>>>>> > On the lack of committers:
> >>>>>>>>> >
> >>>>>>>>> > 1. Ideally, there are some of the current Flink committers who
> >>>>>>>>> have
> >>>>>>>>> >
> >>>>>>>>> > the bandwidth and can help with reviewing PRs and merging them.
> >>>>>>>>> >
> >>>>>>>>> > 2. If that's not an option, it could be a consideration that
> >>>>>>>>> current
> >>>>>>>>> >
> >>>>>>>>> > committers only approve and review PRs, that are approved by
> >>>>>>>>> those who are
> >>>>>>>>> > willing to contribute to Statefun and if the CI passes
> >>>>>>>>> >
> >>>>>>>>> > On the lack of contributors:
> >>>>>>>>> >
> >>>>>>>>> > 3. Next to having this discussion on the Dev and User mailing
> >>>>>>>>> list, we
> >>>>>>>>> >
> >>>>>>>>> > can also create a blog with a call for new contributors on the
> >>>>>>>>> Flink
> >>>>>>>>> > project website, send out some tweets on the Flink / Statefun
> >>>>>>>>> twitter
> >>>>>>>>> > accounts, post messages on Slack etc. In that message, we would
> >>>>>>>>> inform how
> >>>>>>>>> > those that are interested in contributing can start and where
> >>>>>>>>> they could
> >>>>>>>>> > reach out for more information.
> >>>>>>>>> >
> >>>>>>>>> > There's also option 4. where a group of interested people would
> >>>>>>>>> split
> >>>>>>>>> >
> >>>>>>>>> > Statefun from the Flink project and make it a separate top
> level
> >>>>>>>>> project
> >>>>>>>>> > under the Apache Flink umbrella (similar as recently has
> >>>>>>>>> happened with
> >>>>>>>>> > Flink Table Store, which has become Apache Paimon).
> >>>>>>>>> >
> >>>>>>>>> > If we see no improvements in the coming period, we should
> >>>>>>>>> consider
> >>>>>>>>> >
> >>>>>>>>> > sunsetting Statefun and communicate that clearly to the users.
> >>>>>>>>> >
> >>>>>>>>> > I'm looking forward to your thoughts.
> >>>>>>>>> >
> >>>>>>>>> > Best regards,
> >>>>>>>>> >
> >>>>>>>>> > Martijn
> >>>>>>>>> >
> >>>>>>>>> > [1]
> >>>>>>>>> https://nightlies.apache.org/flink/flink-statefun-docs-master/ <
> >>>>>>>>> >
> >>>>>>>>> > https://nightlies.apache.org/flink/flink-statefun-docs-master/
> >
> >>>>>>>>> >
> >>>>>>>>> > [2] https://github.com/apache/flink-statefun <
> >>>>>>>>> >
> >>>>>>>>> > https://github.com/apache/flink-statefun>
> >>>>>>>>> >
> >>>>>>>>> >
> >>>>>>>>> >
> >>>>>>>>> >
> >>>>>>>>> >
> >>>>>>>>>
> >>>>>>>>
>

Re: [DISCUSS] Status of Statefun Project

[Galen Warren <ga...@cvillewarrens.com.INVALID>]

Thanks.

If you were to build a local image, as you suggest, how do you access that
image when building the playground images? All the compilation of
playground code happens inside containers, so local images on the host
aren't available in those containers. Unless I'm missing something?

I've slightly reworked things such that the playground images can be run in
one of two modes -- the default mode, which works like before, and a
"local" mode where locally built code is copied into the build containers
so that it can be accessed during the build. It works fine, you just have
to define a couple of environment variables when running docker-compose to
specify default vs. local mode and what versions of Flink and Statefun
should be referenced, and then you can build a run the local examples
without any additional steps. Does that sound like a reasonable approach?


On Fri, Aug 18, 2023 at 2:17 PM Tzu-Li (Gordon) Tai <tz...@apache.org>
wrote:

> Hi Galen,
>
> > Gordon, is there a trick to running the sample code in
> flink-statefun-playground against yet-unreleased code that I'm missing?
>
> You'd have to locally build an image from the release branch, with a
> temporary image version tag. Then, in the flink-statefun-playground, change
> the image versions in the docker-compose files to use that locally built
> image. IIRC, that's what we have been doing in the past. Admittedly, it's
> pretty manual - I don't think the CI manages this workflow.
>
> Thanks,
> Gordon
>
> On Mon, Aug 14, 2023 at 10:42 AM Galen Warren <ga...@cvillewarrens.com>
> wrote:
>
> > I created a pull request for this: [FLINK-31619] Upgrade Stateful
> > Functions to Flink 1.16.1 by galenwarren · Pull Request #331 ·
> > apache/flink-statefun (github.com)
> > <https://github.com/apache/flink-statefun/pull/331>.
> >
> > JIRA is here: [FLINK-31619] Upgrade Stateful Functions to Flink 1.16.1 -
> > ASF JIRA (apache.org)
> > <https://issues.apache.org/jira/browse/FLINK-31619?filter=-1>.
> >
> > Statefun references 1.16.2, despite the title -- that version has come
> out
> > since the issue was created.
> >
> > I figured out how to run all the playground tests locally, but it took a
> > bit of reworking of the playground setup with respect to Docker;
> > specifically, the Docker contexts used to build the example functions
> > needed to be broadened (i.e. moved up the tree) so that, if needed, local
> > artifacts/code can be accessed from within the containers at build time.
> > Then I made the Docker compose.yml configurable through environment
> > variables to allow for them to run in either the original manner -- i.e.
> > pulling artifacts from public repos -- or in a "local" mode, where
> > artifacts are pulled from local builds.
> >
> > This process is a cleaner if the playground is a subfolder of the
> > flink-statefun project rather than be its own repository
> > (flink-statefun-playground), because then all the relative paths between
> > the playground files and the build artifacts are fixed. So, I'd like to
> > propose to move the playground files, modified as described above, to
> > flink-statefun/playground and retire flink-statefun-playground. I can
> > submit separate PR s those changes if everyone is on board.
> >
> > Also, should I plan to do the same upgrade to handle Flink 1.17.x? It
> > should be easy to do, especially while the 1.16.x upgrade is fresh on my
> > mind.
> >
> > Thanks.
> >
> >
> > On Fri, Aug 11, 2023 at 6:40 PM Galen Warren <ga...@cvillewarrens.com>
> > wrote:
> >
> >> I'm done with the code to make Statefun compatible with Flink 1.16, and
> >> all the tests (including e2e succeed). The required changes were pretty
> >> minimal.
> >>
> >> I'm running into a bit of a chicken/egg problem executing the tests in
> >> flink-statefun-playground
> >> <https://github.com/apache/flink-statefun-playground>, though. That
> >> project seems to assume that all the various Statefun artifacts are
> built
> >> and deployed to the various public repositories already. I've looked
> into
> >> trying to redirect references to local artifacts; however, that's also
> >> tricky since all the building occurs in Docker containers.
> >>
> >> Gordon, is there a trick to running the sample code in
> >> flink-statefun-playground against yet-unreleased code that I'm missing?
> >>
> >> Thanks.
> >>
> >> On Sat, Jun 24, 2023 at 12:40 PM Galen Warren <ga...@cvillewarrens.com>
> >> wrote:
> >>
> >>> Great -- thanks!
> >>>
> >>> I'm going to be out of town for about a week but I'll take a look at
> >>> this when I'm back.
> >>>
> >>> On Tue, Jun 20, 2023 at 8:46 AM Martijn Visser <mv...@confluent.io>
> >>> wrote:
> >>>
> >>>> Hi Galen,
> >>>>
> >>>> Yes, I'll be more than happy to help with Statefun releases.
> >>>>
> >>>> Best regards,
> >>>>
> >>>> Martijn
> >>>>
> >>>> On Tue, Jun 20, 2023 at 2:21 PM Galen Warren <galen@cvillewarrens.com
> >
> >>>> wrote:
> >>>>
> >>>>> Thanks.
> >>>>>
> >>>>> Martijn, to answer your question, I'd need to do a small amount of
> >>>>> work to get a PR ready, but not much. Happy to do it if we're
> deciding to
> >>>>> restart Statefun releases -- are we?
> >>>>>
> >>>>> -- Galen
> >>>>>
> >>>>> On Sat, Jun 17, 2023 at 9:47 AM Tzu-Li (Gordon) Tai <
> >>>>> tzulitai@apache.org> wrote:
> >>>>>
> >>>>>> > Perhaps he could weigh in on whether the combination of automated
> >>>>>> tests plus those smoke tests should be sufficient for testing with
> new
> >>>>>> Flink versions
> >>>>>>
> >>>>>> What we usually did at the bare minimum for new StateFun releases
> was
> >>>>>> the following:
> >>>>>>
> >>>>>>    1. Build tests (including the smoke tests in the e2e module,
> >>>>>>    which covers important tests like exactly-once verification)
> >>>>>>    2. Updating the flink-statefun-playground repo and manually
> >>>>>>    running all language examples there.
> >>>>>>
> >>>>>> If upgrading Flink versions was the only change in the release, I'd
> >>>>>> probably say that this is sufficient.
> >>>>>>
> >>>>>> Best,
> >>>>>> Gordon
> >>>>>>
> >>>>>> On Thu, Jun 15, 2023 at 5:25 AM Martijn Visser <
> >>>>>> martijnvisser@apache.org> wrote:
> >>>>>>
> >>>>>>> Let me know if you have a PR for a Flink update :)
> >>>>>>>
> >>>>>>> On Thu, Jun 8, 2023 at 5:52 PM Galen Warren via user <
> >>>>>>> user@flink.apache.org> wrote:
> >>>>>>>
> >>>>>>>> Thanks Martijn.
> >>>>>>>>
> >>>>>>>> Personally, I'm already using a local fork of Statefun that is
> >>>>>>>> compatible with Flink 1.16.x, so I wouldn't have any need for a
> released
> >>>>>>>> version compatible with 1.15.x. I'd be happy to do the PRs to
> modify
> >>>>>>>> Statefun to work with new versions of Flink as they come along.
> >>>>>>>>
> >>>>>>>> As for testing, Statefun does have unit tests and Gordon also sent
> >>>>>>>> me instructions a while back for how to do some additional smoke
> tests
> >>>>>>>> which are pretty straightforward. Perhaps he could weigh in on
> whether the
> >>>>>>>> combination of automated tests plus those smoke tests should be
> sufficient
> >>>>>>>> for testing with new Flink versions (I believe the answer is yes).
> >>>>>>>>
> >>>>>>>> -- Galen
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Thu, Jun 8, 2023 at 8:01 AM Martijn Visser <
> >>>>>>>> martijnvisser@apache.org> wrote:
> >>>>>>>>
> >>>>>>>>> Hi all,
> >>>>>>>>>
> >>>>>>>>> Apologies for the late reply.
> >>>>>>>>>
> >>>>>>>>> I'm willing to help out with merging requests in Statefun to keep
> >>>>>>>>> them
> >>>>>>>>> compatible with new Flink releases and create new releases. I do
> >>>>>>>>> think that
> >>>>>>>>> validation of the functionality of these releases depends a lot
> on
> >>>>>>>>> those
> >>>>>>>>> who do these compatibility updates, with PMC members helping out
> >>>>>>>>> with the
> >>>>>>>>> formal process.
> >>>>>>>>>
> >>>>>>>>> > Why can't the Apache Software Foundation allow community
> members
> >>>>>>>>> to bring
> >>>>>>>>> it up to date?
> >>>>>>>>>
> >>>>>>>>> There's nothing preventing anyone from reviewing any of the
> >>>>>>>>> current PRs or
> >>>>>>>>> opening new ones. However, none of them are approved [1], so
> >>>>>>>>> there's also
> >>>>>>>>> nothing to merge.
> >>>>>>>>>
> >>>>>>>>> > I believe that there are people and companies on this mailing
> >>>>>>>>> list
> >>>>>>>>> interested in supporting Apache Flink Stateful Functions.
> >>>>>>>>>
> >>>>>>>>> If so, then now is the time to show.
> >>>>>>>>>
> >>>>>>>>> Would there be a preference to create a release with Galen's
> merged
> >>>>>>>>> compatibility update to Flink 1.15.2, or do we want to skip that
> >>>>>>>>> and go
> >>>>>>>>> straight to a newer version?
> >>>>>>>>>
> >>>>>>>>> Best regards,
> >>>>>>>>>
> >>>>>>>>> Martijn
> >>>>>>>>>
> >>>>>>>>> [1]
> >>>>>>>>>
> >>>>>>>>>
> https://github.com/apache/flink-statefun/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved
> >>>>>>>>>
> >>>>>>>>> On Tue, Jun 6, 2023 at 3:55 PM Marco Villalobos <
> >>>>>>>>> mvillalobos@kineteque.com>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>> > Why can't the Apache Software Foundation allow community
> members
> >>>>>>>>> to bring
> >>>>>>>>> > it up to date?
> >>>>>>>>> >
> >>>>>>>>> > What's the process for that?
> >>>>>>>>> >
> >>>>>>>>> > I believe that there are people and companies on this mailing
> >>>>>>>>> list
> >>>>>>>>> > interested in supporting Apache Flink Stateful Functions.
> >>>>>>>>> >
> >>>>>>>>> > You already had two people on this thread express interest.
> >>>>>>>>> >
> >>>>>>>>> > At the very least, we could keep the library versions up to
> date.
> >>>>>>>>> >
> >>>>>>>>> > There are only a small list of new features that might be
> >>>>>>>>> worthwhile:
> >>>>>>>>> >
> >>>>>>>>> > 1. event time processing
> >>>>>>>>> > 2. state rest api
> >>>>>>>>> >
> >>>>>>>>> >
> >>>>>>>>> > On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <
> chesnay@apache.org>
> >>>>>>>>> wrote:
> >>>>>>>>> >
> >>>>>>>>> > If you were to fork it *and want to redistribute it* then the
> >>>>>>>>> short
> >>>>>>>>> > version is that
> >>>>>>>>> >
> >>>>>>>>> >    1. you have to adhere to the Apache licensing requirements
> >>>>>>>>> >    2. you have to make it clear that your fork does not belong
> >>>>>>>>> to the
> >>>>>>>>> >    Apache Flink project. (Trademarks and all that)
> >>>>>>>>> >
> >>>>>>>>> > Neither should be significant hurdles (there should also be
> >>>>>>>>> plenty of
> >>>>>>>>> > online resources regarding 1), and if you do this then you can
> >>>>>>>>> freely share
> >>>>>>>>> > your fork with others.
> >>>>>>>>> >
> >>>>>>>>> > I've also pinged Martijn to take a look at this thread.
> >>>>>>>>> > To my knowledge the project hasn't decided anything yet.
> >>>>>>>>> >
> >>>>>>>>> > On 27/05/2023 04:05, Galen Warren wrote:
> >>>>>>>>> >
> >>>>>>>>> > Ok, I get it. No interest.
> >>>>>>>>> >
> >>>>>>>>> > If this project is being abandoned, I guess I'll work with my
> >>>>>>>>> own fork. Is
> >>>>>>>>> > there anything I should consider here? Can I share it with
> other
> >>>>>>>>> people who
> >>>>>>>>> > use this project?
> >>>>>>>>> >
> >>>>>>>>> > On Tue, May 16, 2023 at 10:50 AM Galen Warren <
> >>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
> >>>>>>>>> > wrote:
> >>>>>>>>> >
> >>>>>>>>> >
> >>>>>>>>> > Hi Martijn, since you opened this discussion thread, I'm
> curious
> >>>>>>>>> what your
> >>>>>>>>> > thoughts are in light of the responses? Thanks.
> >>>>>>>>> >
> >>>>>>>>> > On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <
> >>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
> >>>>>>>>> > wrote:
> >>>>>>>>> >
> >>>>>>>>> >
> >>>>>>>>> > I use Apache Flink for stream processing, and StateFun as a
> >>>>>>>>> hand-off
> >>>>>>>>> >
> >>>>>>>>> > point for the rest of the application.
> >>>>>>>>> > It serves well as a bridge between a Flink Streaming job and
> >>>>>>>>> > micro-services.
> >>>>>>>>> >
> >>>>>>>>> > This is essentially how I use it as well, and I would also be
> >>>>>>>>> sad to see
> >>>>>>>>> > it sunsetted. It works well; I don't know that there is a lot
> of
> >>>>>>>>> new
> >>>>>>>>> > development required, but if there are no new Statefun
> releases,
> >>>>>>>>> then
> >>>>>>>>> > Statefun can only be used with older Flink versions.
> >>>>>>>>> >
> >>>>>>>>> > On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
> >>>>>>>>> mvillalobos@kineteque.com> wrote:
> >>>>>>>>> >
> >>>>>>>>> >
> >>>>>>>>> > I am currently using Stateful Functions in my application.
> >>>>>>>>> >
> >>>>>>>>> > I use Apache Flink for stream processing, and StateFun as a
> >>>>>>>>> hand-off
> >>>>>>>>> > point for the rest of the application.
> >>>>>>>>> > It serves well as a bridge between a Flink Streaming job and
> >>>>>>>>> > micro-services.
> >>>>>>>>> >
> >>>>>>>>> > I would be disappointed if StateFun was sunsetted.  Its a good
> >>>>>>>>> idea.
> >>>>>>>>> >
> >>>>>>>>> > If there is anything I can do to help, as a contributor
> perhaps,
> >>>>>>>>> please
> >>>>>>>>> > let me know.
> >>>>>>>>> >
> >>>>>>>>> >
> >>>>>>>>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <
> >>>>>>>>> martijnvisser@apache.org> <ma...@apache.org>
> >>>>>>>>> >
> >>>>>>>>> > wrote:
> >>>>>>>>> >
> >>>>>>>>> > Hi everyone,
> >>>>>>>>> >
> >>>>>>>>> > I want to open a discussion on the status of the Statefun
> >>>>>>>>> Project [1]
> >>>>>>>>> >
> >>>>>>>>> > in Apache Flink. As you might have noticed, there hasn't been
> >>>>>>>>> much
> >>>>>>>>> > development over the past months in the Statefun repository
> [2].
> >>>>>>>>> There is
> >>>>>>>>> > currently a lack of active contributors and committers who are
> >>>>>>>>> able to help
> >>>>>>>>> > with the maintenance of the project.
> >>>>>>>>> >
> >>>>>>>>> > In order to improve the situation, we need to solve the lack of
> >>>>>>>>> >
> >>>>>>>>> > committers and the lack of contributors.
> >>>>>>>>> >
> >>>>>>>>> > On the lack of committers:
> >>>>>>>>> >
> >>>>>>>>> > 1. Ideally, there are some of the current Flink committers who
> >>>>>>>>> have
> >>>>>>>>> >
> >>>>>>>>> > the bandwidth and can help with reviewing PRs and merging them.
> >>>>>>>>> >
> >>>>>>>>> > 2. If that's not an option, it could be a consideration that
> >>>>>>>>> current
> >>>>>>>>> >
> >>>>>>>>> > committers only approve and review PRs, that are approved by
> >>>>>>>>> those who are
> >>>>>>>>> > willing to contribute to Statefun and if the CI passes
> >>>>>>>>> >
> >>>>>>>>> > On the lack of contributors:
> >>>>>>>>> >
> >>>>>>>>> > 3. Next to having this discussion on the Dev and User mailing
> >>>>>>>>> list, we
> >>>>>>>>> >
> >>>>>>>>> > can also create a blog with a call for new contributors on the
> >>>>>>>>> Flink
> >>>>>>>>> > project website, send out some tweets on the Flink / Statefun
> >>>>>>>>> twitter
> >>>>>>>>> > accounts, post messages on Slack etc. In that message, we would
> >>>>>>>>> inform how
> >>>>>>>>> > those that are interested in contributing can start and where
> >>>>>>>>> they could
> >>>>>>>>> > reach out for more information.
> >>>>>>>>> >
> >>>>>>>>> > There's also option 4. where a group of interested people would
> >>>>>>>>> split
> >>>>>>>>> >
> >>>>>>>>> > Statefun from the Flink project and make it a separate top
> level
> >>>>>>>>> project
> >>>>>>>>> > under the Apache Flink umbrella (similar as recently has
> >>>>>>>>> happened with
> >>>>>>>>> > Flink Table Store, which has become Apache Paimon).
> >>>>>>>>> >
> >>>>>>>>> > If we see no improvements in the coming period, we should
> >>>>>>>>> consider
> >>>>>>>>> >
> >>>>>>>>> > sunsetting Statefun and communicate that clearly to the users.
> >>>>>>>>> >
> >>>>>>>>> > I'm looking forward to your thoughts.
> >>>>>>>>> >
> >>>>>>>>> > Best regards,
> >>>>>>>>> >
> >>>>>>>>> > Martijn
> >>>>>>>>> >
> >>>>>>>>> > [1]
> >>>>>>>>> https://nightlies.apache.org/flink/flink-statefun-docs-master/ <
> >>>>>>>>> >
> >>>>>>>>> > https://nightlies.apache.org/flink/flink-statefun-docs-master/
> >
> >>>>>>>>> >
> >>>>>>>>> > [2] https://github.com/apache/flink-statefun <
> >>>>>>>>> >
> >>>>>>>>> > https://github.com/apache/flink-statefun>
> >>>>>>>>> >
> >>>>>>>>> >
> >>>>>>>>> >
> >>>>>>>>> >
> >>>>>>>>> >
> >>>>>>>>>
> >>>>>>>>
>

Re: [DISCUSS] Status of Statefun Project

["Tzu-Li (Gordon) Tai" <tz...@apache.org>]

Hi Galen,

> Gordon, is there a trick to running the sample code in
flink-statefun-playground against yet-unreleased code that I'm missing?

You'd have to locally build an image from the release branch, with a
temporary image version tag. Then, in the flink-statefun-playground, change
the image versions in the docker-compose files to use that locally built
image. IIRC, that's what we have been doing in the past. Admittedly, it's
pretty manual - I don't think the CI manages this workflow.

Thanks,
Gordon

On Mon, Aug 14, 2023 at 10:42 AM Galen Warren <ga...@cvillewarrens.com>
wrote:

> I created a pull request for this: [FLINK-31619] Upgrade Stateful
> Functions to Flink 1.16.1 by galenwarren · Pull Request #331 ·
> apache/flink-statefun (github.com)
> <https://github.com/apache/flink-statefun/pull/331>.
>
> JIRA is here: [FLINK-31619] Upgrade Stateful Functions to Flink 1.16.1 -
> ASF JIRA (apache.org)
> <https://issues.apache.org/jira/browse/FLINK-31619?filter=-1>.
>
> Statefun references 1.16.2, despite the title -- that version has come out
> since the issue was created.
>
> I figured out how to run all the playground tests locally, but it took a
> bit of reworking of the playground setup with respect to Docker;
> specifically, the Docker contexts used to build the example functions
> needed to be broadened (i.e. moved up the tree) so that, if needed, local
> artifacts/code can be accessed from within the containers at build time.
> Then I made the Docker compose.yml configurable through environment
> variables to allow for them to run in either the original manner -- i.e.
> pulling artifacts from public repos -- or in a "local" mode, where
> artifacts are pulled from local builds.
>
> This process is a cleaner if the playground is a subfolder of the
> flink-statefun project rather than be its own repository
> (flink-statefun-playground), because then all the relative paths between
> the playground files and the build artifacts are fixed. So, I'd like to
> propose to move the playground files, modified as described above, to
> flink-statefun/playground and retire flink-statefun-playground. I can
> submit separate PR s those changes if everyone is on board.
>
> Also, should I plan to do the same upgrade to handle Flink 1.17.x? It
> should be easy to do, especially while the 1.16.x upgrade is fresh on my
> mind.
>
> Thanks.
>
>
> On Fri, Aug 11, 2023 at 6:40 PM Galen Warren <ga...@cvillewarrens.com>
> wrote:
>
>> I'm done with the code to make Statefun compatible with Flink 1.16, and
>> all the tests (including e2e succeed). The required changes were pretty
>> minimal.
>>
>> I'm running into a bit of a chicken/egg problem executing the tests in
>> flink-statefun-playground
>> <https://github.com/apache/flink-statefun-playground>, though. That
>> project seems to assume that all the various Statefun artifacts are built
>> and deployed to the various public repositories already. I've looked into
>> trying to redirect references to local artifacts; however, that's also
>> tricky since all the building occurs in Docker containers.
>>
>> Gordon, is there a trick to running the sample code in
>> flink-statefun-playground against yet-unreleased code that I'm missing?
>>
>> Thanks.
>>
>> On Sat, Jun 24, 2023 at 12:40 PM Galen Warren <ga...@cvillewarrens.com>
>> wrote:
>>
>>> Great -- thanks!
>>>
>>> I'm going to be out of town for about a week but I'll take a look at
>>> this when I'm back.
>>>
>>> On Tue, Jun 20, 2023 at 8:46 AM Martijn Visser <mv...@confluent.io>
>>> wrote:
>>>
>>>> Hi Galen,
>>>>
>>>> Yes, I'll be more than happy to help with Statefun releases.
>>>>
>>>> Best regards,
>>>>
>>>> Martijn
>>>>
>>>> On Tue, Jun 20, 2023 at 2:21 PM Galen Warren <ga...@cvillewarrens.com>
>>>> wrote:
>>>>
>>>>> Thanks.
>>>>>
>>>>> Martijn, to answer your question, I'd need to do a small amount of
>>>>> work to get a PR ready, but not much. Happy to do it if we're deciding to
>>>>> restart Statefun releases -- are we?
>>>>>
>>>>> -- Galen
>>>>>
>>>>> On Sat, Jun 17, 2023 at 9:47 AM Tzu-Li (Gordon) Tai <
>>>>> tzulitai@apache.org> wrote:
>>>>>
>>>>>> > Perhaps he could weigh in on whether the combination of automated
>>>>>> tests plus those smoke tests should be sufficient for testing with new
>>>>>> Flink versions
>>>>>>
>>>>>> What we usually did at the bare minimum for new StateFun releases was
>>>>>> the following:
>>>>>>
>>>>>>    1. Build tests (including the smoke tests in the e2e module,
>>>>>>    which covers important tests like exactly-once verification)
>>>>>>    2. Updating the flink-statefun-playground repo and manually
>>>>>>    running all language examples there.
>>>>>>
>>>>>> If upgrading Flink versions was the only change in the release, I'd
>>>>>> probably say that this is sufficient.
>>>>>>
>>>>>> Best,
>>>>>> Gordon
>>>>>>
>>>>>> On Thu, Jun 15, 2023 at 5:25 AM Martijn Visser <
>>>>>> martijnvisser@apache.org> wrote:
>>>>>>
>>>>>>> Let me know if you have a PR for a Flink update :)
>>>>>>>
>>>>>>> On Thu, Jun 8, 2023 at 5:52 PM Galen Warren via user <
>>>>>>> user@flink.apache.org> wrote:
>>>>>>>
>>>>>>>> Thanks Martijn.
>>>>>>>>
>>>>>>>> Personally, I'm already using a local fork of Statefun that is
>>>>>>>> compatible with Flink 1.16.x, so I wouldn't have any need for a released
>>>>>>>> version compatible with 1.15.x. I'd be happy to do the PRs to modify
>>>>>>>> Statefun to work with new versions of Flink as they come along.
>>>>>>>>
>>>>>>>> As for testing, Statefun does have unit tests and Gordon also sent
>>>>>>>> me instructions a while back for how to do some additional smoke tests
>>>>>>>> which are pretty straightforward. Perhaps he could weigh in on whether the
>>>>>>>> combination of automated tests plus those smoke tests should be sufficient
>>>>>>>> for testing with new Flink versions (I believe the answer is yes).
>>>>>>>>
>>>>>>>> -- Galen
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Jun 8, 2023 at 8:01 AM Martijn Visser <
>>>>>>>> martijnvisser@apache.org> wrote:
>>>>>>>>
>>>>>>>>> Hi all,
>>>>>>>>>
>>>>>>>>> Apologies for the late reply.
>>>>>>>>>
>>>>>>>>> I'm willing to help out with merging requests in Statefun to keep
>>>>>>>>> them
>>>>>>>>> compatible with new Flink releases and create new releases. I do
>>>>>>>>> think that
>>>>>>>>> validation of the functionality of these releases depends a lot on
>>>>>>>>> those
>>>>>>>>> who do these compatibility updates, with PMC members helping out
>>>>>>>>> with the
>>>>>>>>> formal process.
>>>>>>>>>
>>>>>>>>> > Why can't the Apache Software Foundation allow community members
>>>>>>>>> to bring
>>>>>>>>> it up to date?
>>>>>>>>>
>>>>>>>>> There's nothing preventing anyone from reviewing any of the
>>>>>>>>> current PRs or
>>>>>>>>> opening new ones. However, none of them are approved [1], so
>>>>>>>>> there's also
>>>>>>>>> nothing to merge.
>>>>>>>>>
>>>>>>>>> > I believe that there are people and companies on this mailing
>>>>>>>>> list
>>>>>>>>> interested in supporting Apache Flink Stateful Functions.
>>>>>>>>>
>>>>>>>>> If so, then now is the time to show.
>>>>>>>>>
>>>>>>>>> Would there be a preference to create a release with Galen's merged
>>>>>>>>> compatibility update to Flink 1.15.2, or do we want to skip that
>>>>>>>>> and go
>>>>>>>>> straight to a newer version?
>>>>>>>>>
>>>>>>>>> Best regards,
>>>>>>>>>
>>>>>>>>> Martijn
>>>>>>>>>
>>>>>>>>> [1]
>>>>>>>>>
>>>>>>>>> https://github.com/apache/flink-statefun/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved
>>>>>>>>>
>>>>>>>>> On Tue, Jun 6, 2023 at 3:55 PM Marco Villalobos <
>>>>>>>>> mvillalobos@kineteque.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> > Why can't the Apache Software Foundation allow community members
>>>>>>>>> to bring
>>>>>>>>> > it up to date?
>>>>>>>>> >
>>>>>>>>> > What's the process for that?
>>>>>>>>> >
>>>>>>>>> > I believe that there are people and companies on this mailing
>>>>>>>>> list
>>>>>>>>> > interested in supporting Apache Flink Stateful Functions.
>>>>>>>>> >
>>>>>>>>> > You already had two people on this thread express interest.
>>>>>>>>> >
>>>>>>>>> > At the very least, we could keep the library versions up to date.
>>>>>>>>> >
>>>>>>>>> > There are only a small list of new features that might be
>>>>>>>>> worthwhile:
>>>>>>>>> >
>>>>>>>>> > 1. event time processing
>>>>>>>>> > 2. state rest api
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> > On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <ch...@apache.org>
>>>>>>>>> wrote:
>>>>>>>>> >
>>>>>>>>> > If you were to fork it *and want to redistribute it* then the
>>>>>>>>> short
>>>>>>>>> > version is that
>>>>>>>>> >
>>>>>>>>> >    1. you have to adhere to the Apache licensing requirements
>>>>>>>>> >    2. you have to make it clear that your fork does not belong
>>>>>>>>> to the
>>>>>>>>> >    Apache Flink project. (Trademarks and all that)
>>>>>>>>> >
>>>>>>>>> > Neither should be significant hurdles (there should also be
>>>>>>>>> plenty of
>>>>>>>>> > online resources regarding 1), and if you do this then you can
>>>>>>>>> freely share
>>>>>>>>> > your fork with others.
>>>>>>>>> >
>>>>>>>>> > I've also pinged Martijn to take a look at this thread.
>>>>>>>>> > To my knowledge the project hasn't decided anything yet.
>>>>>>>>> >
>>>>>>>>> > On 27/05/2023 04:05, Galen Warren wrote:
>>>>>>>>> >
>>>>>>>>> > Ok, I get it. No interest.
>>>>>>>>> >
>>>>>>>>> > If this project is being abandoned, I guess I'll work with my
>>>>>>>>> own fork. Is
>>>>>>>>> > there anything I should consider here? Can I share it with other
>>>>>>>>> people who
>>>>>>>>> > use this project?
>>>>>>>>> >
>>>>>>>>> > On Tue, May 16, 2023 at 10:50 AM Galen Warren <
>>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
>>>>>>>>> > wrote:
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> > Hi Martijn, since you opened this discussion thread, I'm curious
>>>>>>>>> what your
>>>>>>>>> > thoughts are in light of the responses? Thanks.
>>>>>>>>> >
>>>>>>>>> > On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <
>>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
>>>>>>>>> > wrote:
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> > I use Apache Flink for stream processing, and StateFun as a
>>>>>>>>> hand-off
>>>>>>>>> >
>>>>>>>>> > point for the rest of the application.
>>>>>>>>> > It serves well as a bridge between a Flink Streaming job and
>>>>>>>>> > micro-services.
>>>>>>>>> >
>>>>>>>>> > This is essentially how I use it as well, and I would also be
>>>>>>>>> sad to see
>>>>>>>>> > it sunsetted. It works well; I don't know that there is a lot of
>>>>>>>>> new
>>>>>>>>> > development required, but if there are no new Statefun releases,
>>>>>>>>> then
>>>>>>>>> > Statefun can only be used with older Flink versions.
>>>>>>>>> >
>>>>>>>>> > On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
>>>>>>>>> mvillalobos@kineteque.com> wrote:
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> > I am currently using Stateful Functions in my application.
>>>>>>>>> >
>>>>>>>>> > I use Apache Flink for stream processing, and StateFun as a
>>>>>>>>> hand-off
>>>>>>>>> > point for the rest of the application.
>>>>>>>>> > It serves well as a bridge between a Flink Streaming job and
>>>>>>>>> > micro-services.
>>>>>>>>> >
>>>>>>>>> > I would be disappointed if StateFun was sunsetted.  Its a good
>>>>>>>>> idea.
>>>>>>>>> >
>>>>>>>>> > If there is anything I can do to help, as a contributor perhaps,
>>>>>>>>> please
>>>>>>>>> > let me know.
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <
>>>>>>>>> martijnvisser@apache.org> <ma...@apache.org>
>>>>>>>>> >
>>>>>>>>> > wrote:
>>>>>>>>> >
>>>>>>>>> > Hi everyone,
>>>>>>>>> >
>>>>>>>>> > I want to open a discussion on the status of the Statefun
>>>>>>>>> Project [1]
>>>>>>>>> >
>>>>>>>>> > in Apache Flink. As you might have noticed, there hasn't been
>>>>>>>>> much
>>>>>>>>> > development over the past months in the Statefun repository [2].
>>>>>>>>> There is
>>>>>>>>> > currently a lack of active contributors and committers who are
>>>>>>>>> able to help
>>>>>>>>> > with the maintenance of the project.
>>>>>>>>> >
>>>>>>>>> > In order to improve the situation, we need to solve the lack of
>>>>>>>>> >
>>>>>>>>> > committers and the lack of contributors.
>>>>>>>>> >
>>>>>>>>> > On the lack of committers:
>>>>>>>>> >
>>>>>>>>> > 1. Ideally, there are some of the current Flink committers who
>>>>>>>>> have
>>>>>>>>> >
>>>>>>>>> > the bandwidth and can help with reviewing PRs and merging them.
>>>>>>>>> >
>>>>>>>>> > 2. If that's not an option, it could be a consideration that
>>>>>>>>> current
>>>>>>>>> >
>>>>>>>>> > committers only approve and review PRs, that are approved by
>>>>>>>>> those who are
>>>>>>>>> > willing to contribute to Statefun and if the CI passes
>>>>>>>>> >
>>>>>>>>> > On the lack of contributors:
>>>>>>>>> >
>>>>>>>>> > 3. Next to having this discussion on the Dev and User mailing
>>>>>>>>> list, we
>>>>>>>>> >
>>>>>>>>> > can also create a blog with a call for new contributors on the
>>>>>>>>> Flink
>>>>>>>>> > project website, send out some tweets on the Flink / Statefun
>>>>>>>>> twitter
>>>>>>>>> > accounts, post messages on Slack etc. In that message, we would
>>>>>>>>> inform how
>>>>>>>>> > those that are interested in contributing can start and where
>>>>>>>>> they could
>>>>>>>>> > reach out for more information.
>>>>>>>>> >
>>>>>>>>> > There's also option 4. where a group of interested people would
>>>>>>>>> split
>>>>>>>>> >
>>>>>>>>> > Statefun from the Flink project and make it a separate top level
>>>>>>>>> project
>>>>>>>>> > under the Apache Flink umbrella (similar as recently has
>>>>>>>>> happened with
>>>>>>>>> > Flink Table Store, which has become Apache Paimon).
>>>>>>>>> >
>>>>>>>>> > If we see no improvements in the coming period, we should
>>>>>>>>> consider
>>>>>>>>> >
>>>>>>>>> > sunsetting Statefun and communicate that clearly to the users.
>>>>>>>>> >
>>>>>>>>> > I'm looking forward to your thoughts.
>>>>>>>>> >
>>>>>>>>> > Best regards,
>>>>>>>>> >
>>>>>>>>> > Martijn
>>>>>>>>> >
>>>>>>>>> > [1]
>>>>>>>>> https://nightlies.apache.org/flink/flink-statefun-docs-master/ <
>>>>>>>>> >
>>>>>>>>> > https://nightlies.apache.org/flink/flink-statefun-docs-master/>
>>>>>>>>> >
>>>>>>>>> > [2] https://github.com/apache/flink-statefun <
>>>>>>>>> >
>>>>>>>>> > https://github.com/apache/flink-statefun>
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>>
>>>>>>>>

Re: [DISCUSS] Status of Statefun Project

["Tzu-Li (Gordon) Tai" <tz...@apache.org>]

Hi Galen,

> Gordon, is there a trick to running the sample code in
flink-statefun-playground against yet-unreleased code that I'm missing?

You'd have to locally build an image from the release branch, with a
temporary image version tag. Then, in the flink-statefun-playground, change
the image versions in the docker-compose files to use that locally built
image. IIRC, that's what we have been doing in the past. Admittedly, it's
pretty manual - I don't think the CI manages this workflow.

Thanks,
Gordon

On Mon, Aug 14, 2023 at 10:42 AM Galen Warren <ga...@cvillewarrens.com>
wrote:

> I created a pull request for this: [FLINK-31619] Upgrade Stateful
> Functions to Flink 1.16.1 by galenwarren · Pull Request #331 ·
> apache/flink-statefun (github.com)
> <https://github.com/apache/flink-statefun/pull/331>.
>
> JIRA is here: [FLINK-31619] Upgrade Stateful Functions to Flink 1.16.1 -
> ASF JIRA (apache.org)
> <https://issues.apache.org/jira/browse/FLINK-31619?filter=-1>.
>
> Statefun references 1.16.2, despite the title -- that version has come out
> since the issue was created.
>
> I figured out how to run all the playground tests locally, but it took a
> bit of reworking of the playground setup with respect to Docker;
> specifically, the Docker contexts used to build the example functions
> needed to be broadened (i.e. moved up the tree) so that, if needed, local
> artifacts/code can be accessed from within the containers at build time.
> Then I made the Docker compose.yml configurable through environment
> variables to allow for them to run in either the original manner -- i.e.
> pulling artifacts from public repos -- or in a "local" mode, where
> artifacts are pulled from local builds.
>
> This process is a cleaner if the playground is a subfolder of the
> flink-statefun project rather than be its own repository
> (flink-statefun-playground), because then all the relative paths between
> the playground files and the build artifacts are fixed. So, I'd like to
> propose to move the playground files, modified as described above, to
> flink-statefun/playground and retire flink-statefun-playground. I can
> submit separate PR s those changes if everyone is on board.
>
> Also, should I plan to do the same upgrade to handle Flink 1.17.x? It
> should be easy to do, especially while the 1.16.x upgrade is fresh on my
> mind.
>
> Thanks.
>
>
> On Fri, Aug 11, 2023 at 6:40 PM Galen Warren <ga...@cvillewarrens.com>
> wrote:
>
>> I'm done with the code to make Statefun compatible with Flink 1.16, and
>> all the tests (including e2e succeed). The required changes were pretty
>> minimal.
>>
>> I'm running into a bit of a chicken/egg problem executing the tests in
>> flink-statefun-playground
>> <https://github.com/apache/flink-statefun-playground>, though. That
>> project seems to assume that all the various Statefun artifacts are built
>> and deployed to the various public repositories already. I've looked into
>> trying to redirect references to local artifacts; however, that's also
>> tricky since all the building occurs in Docker containers.
>>
>> Gordon, is there a trick to running the sample code in
>> flink-statefun-playground against yet-unreleased code that I'm missing?
>>
>> Thanks.
>>
>> On Sat, Jun 24, 2023 at 12:40 PM Galen Warren <ga...@cvillewarrens.com>
>> wrote:
>>
>>> Great -- thanks!
>>>
>>> I'm going to be out of town for about a week but I'll take a look at
>>> this when I'm back.
>>>
>>> On Tue, Jun 20, 2023 at 8:46 AM Martijn Visser <mv...@confluent.io>
>>> wrote:
>>>
>>>> Hi Galen,
>>>>
>>>> Yes, I'll be more than happy to help with Statefun releases.
>>>>
>>>> Best regards,
>>>>
>>>> Martijn
>>>>
>>>> On Tue, Jun 20, 2023 at 2:21 PM Galen Warren <ga...@cvillewarrens.com>
>>>> wrote:
>>>>
>>>>> Thanks.
>>>>>
>>>>> Martijn, to answer your question, I'd need to do a small amount of
>>>>> work to get a PR ready, but not much. Happy to do it if we're deciding to
>>>>> restart Statefun releases -- are we?
>>>>>
>>>>> -- Galen
>>>>>
>>>>> On Sat, Jun 17, 2023 at 9:47 AM Tzu-Li (Gordon) Tai <
>>>>> tzulitai@apache.org> wrote:
>>>>>
>>>>>> > Perhaps he could weigh in on whether the combination of automated
>>>>>> tests plus those smoke tests should be sufficient for testing with new
>>>>>> Flink versions
>>>>>>
>>>>>> What we usually did at the bare minimum for new StateFun releases was
>>>>>> the following:
>>>>>>
>>>>>>    1. Build tests (including the smoke tests in the e2e module,
>>>>>>    which covers important tests like exactly-once verification)
>>>>>>    2. Updating the flink-statefun-playground repo and manually
>>>>>>    running all language examples there.
>>>>>>
>>>>>> If upgrading Flink versions was the only change in the release, I'd
>>>>>> probably say that this is sufficient.
>>>>>>
>>>>>> Best,
>>>>>> Gordon
>>>>>>
>>>>>> On Thu, Jun 15, 2023 at 5:25 AM Martijn Visser <
>>>>>> martijnvisser@apache.org> wrote:
>>>>>>
>>>>>>> Let me know if you have a PR for a Flink update :)
>>>>>>>
>>>>>>> On Thu, Jun 8, 2023 at 5:52 PM Galen Warren via user <
>>>>>>> user@flink.apache.org> wrote:
>>>>>>>
>>>>>>>> Thanks Martijn.
>>>>>>>>
>>>>>>>> Personally, I'm already using a local fork of Statefun that is
>>>>>>>> compatible with Flink 1.16.x, so I wouldn't have any need for a released
>>>>>>>> version compatible with 1.15.x. I'd be happy to do the PRs to modify
>>>>>>>> Statefun to work with new versions of Flink as they come along.
>>>>>>>>
>>>>>>>> As for testing, Statefun does have unit tests and Gordon also sent
>>>>>>>> me instructions a while back for how to do some additional smoke tests
>>>>>>>> which are pretty straightforward. Perhaps he could weigh in on whether the
>>>>>>>> combination of automated tests plus those smoke tests should be sufficient
>>>>>>>> for testing with new Flink versions (I believe the answer is yes).
>>>>>>>>
>>>>>>>> -- Galen
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Jun 8, 2023 at 8:01 AM Martijn Visser <
>>>>>>>> martijnvisser@apache.org> wrote:
>>>>>>>>
>>>>>>>>> Hi all,
>>>>>>>>>
>>>>>>>>> Apologies for the late reply.
>>>>>>>>>
>>>>>>>>> I'm willing to help out with merging requests in Statefun to keep
>>>>>>>>> them
>>>>>>>>> compatible with new Flink releases and create new releases. I do
>>>>>>>>> think that
>>>>>>>>> validation of the functionality of these releases depends a lot on
>>>>>>>>> those
>>>>>>>>> who do these compatibility updates, with PMC members helping out
>>>>>>>>> with the
>>>>>>>>> formal process.
>>>>>>>>>
>>>>>>>>> > Why can't the Apache Software Foundation allow community members
>>>>>>>>> to bring
>>>>>>>>> it up to date?
>>>>>>>>>
>>>>>>>>> There's nothing preventing anyone from reviewing any of the
>>>>>>>>> current PRs or
>>>>>>>>> opening new ones. However, none of them are approved [1], so
>>>>>>>>> there's also
>>>>>>>>> nothing to merge.
>>>>>>>>>
>>>>>>>>> > I believe that there are people and companies on this mailing
>>>>>>>>> list
>>>>>>>>> interested in supporting Apache Flink Stateful Functions.
>>>>>>>>>
>>>>>>>>> If so, then now is the time to show.
>>>>>>>>>
>>>>>>>>> Would there be a preference to create a release with Galen's merged
>>>>>>>>> compatibility update to Flink 1.15.2, or do we want to skip that
>>>>>>>>> and go
>>>>>>>>> straight to a newer version?
>>>>>>>>>
>>>>>>>>> Best regards,
>>>>>>>>>
>>>>>>>>> Martijn
>>>>>>>>>
>>>>>>>>> [1]
>>>>>>>>>
>>>>>>>>> https://github.com/apache/flink-statefun/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved
>>>>>>>>>
>>>>>>>>> On Tue, Jun 6, 2023 at 3:55 PM Marco Villalobos <
>>>>>>>>> mvillalobos@kineteque.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> > Why can't the Apache Software Foundation allow community members
>>>>>>>>> to bring
>>>>>>>>> > it up to date?
>>>>>>>>> >
>>>>>>>>> > What's the process for that?
>>>>>>>>> >
>>>>>>>>> > I believe that there are people and companies on this mailing
>>>>>>>>> list
>>>>>>>>> > interested in supporting Apache Flink Stateful Functions.
>>>>>>>>> >
>>>>>>>>> > You already had two people on this thread express interest.
>>>>>>>>> >
>>>>>>>>> > At the very least, we could keep the library versions up to date.
>>>>>>>>> >
>>>>>>>>> > There are only a small list of new features that might be
>>>>>>>>> worthwhile:
>>>>>>>>> >
>>>>>>>>> > 1. event time processing
>>>>>>>>> > 2. state rest api
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> > On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <ch...@apache.org>
>>>>>>>>> wrote:
>>>>>>>>> >
>>>>>>>>> > If you were to fork it *and want to redistribute it* then the
>>>>>>>>> short
>>>>>>>>> > version is that
>>>>>>>>> >
>>>>>>>>> >    1. you have to adhere to the Apache licensing requirements
>>>>>>>>> >    2. you have to make it clear that your fork does not belong
>>>>>>>>> to the
>>>>>>>>> >    Apache Flink project. (Trademarks and all that)
>>>>>>>>> >
>>>>>>>>> > Neither should be significant hurdles (there should also be
>>>>>>>>> plenty of
>>>>>>>>> > online resources regarding 1), and if you do this then you can
>>>>>>>>> freely share
>>>>>>>>> > your fork with others.
>>>>>>>>> >
>>>>>>>>> > I've also pinged Martijn to take a look at this thread.
>>>>>>>>> > To my knowledge the project hasn't decided anything yet.
>>>>>>>>> >
>>>>>>>>> > On 27/05/2023 04:05, Galen Warren wrote:
>>>>>>>>> >
>>>>>>>>> > Ok, I get it. No interest.
>>>>>>>>> >
>>>>>>>>> > If this project is being abandoned, I guess I'll work with my
>>>>>>>>> own fork. Is
>>>>>>>>> > there anything I should consider here? Can I share it with other
>>>>>>>>> people who
>>>>>>>>> > use this project?
>>>>>>>>> >
>>>>>>>>> > On Tue, May 16, 2023 at 10:50 AM Galen Warren <
>>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
>>>>>>>>> > wrote:
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> > Hi Martijn, since you opened this discussion thread, I'm curious
>>>>>>>>> what your
>>>>>>>>> > thoughts are in light of the responses? Thanks.
>>>>>>>>> >
>>>>>>>>> > On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <
>>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
>>>>>>>>> > wrote:
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> > I use Apache Flink for stream processing, and StateFun as a
>>>>>>>>> hand-off
>>>>>>>>> >
>>>>>>>>> > point for the rest of the application.
>>>>>>>>> > It serves well as a bridge between a Flink Streaming job and
>>>>>>>>> > micro-services.
>>>>>>>>> >
>>>>>>>>> > This is essentially how I use it as well, and I would also be
>>>>>>>>> sad to see
>>>>>>>>> > it sunsetted. It works well; I don't know that there is a lot of
>>>>>>>>> new
>>>>>>>>> > development required, but if there are no new Statefun releases,
>>>>>>>>> then
>>>>>>>>> > Statefun can only be used with older Flink versions.
>>>>>>>>> >
>>>>>>>>> > On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
>>>>>>>>> mvillalobos@kineteque.com> wrote:
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> > I am currently using Stateful Functions in my application.
>>>>>>>>> >
>>>>>>>>> > I use Apache Flink for stream processing, and StateFun as a
>>>>>>>>> hand-off
>>>>>>>>> > point for the rest of the application.
>>>>>>>>> > It serves well as a bridge between a Flink Streaming job and
>>>>>>>>> > micro-services.
>>>>>>>>> >
>>>>>>>>> > I would be disappointed if StateFun was sunsetted.  Its a good
>>>>>>>>> idea.
>>>>>>>>> >
>>>>>>>>> > If there is anything I can do to help, as a contributor perhaps,
>>>>>>>>> please
>>>>>>>>> > let me know.
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <
>>>>>>>>> martijnvisser@apache.org> <ma...@apache.org>
>>>>>>>>> >
>>>>>>>>> > wrote:
>>>>>>>>> >
>>>>>>>>> > Hi everyone,
>>>>>>>>> >
>>>>>>>>> > I want to open a discussion on the status of the Statefun
>>>>>>>>> Project [1]
>>>>>>>>> >
>>>>>>>>> > in Apache Flink. As you might have noticed, there hasn't been
>>>>>>>>> much
>>>>>>>>> > development over the past months in the Statefun repository [2].
>>>>>>>>> There is
>>>>>>>>> > currently a lack of active contributors and committers who are
>>>>>>>>> able to help
>>>>>>>>> > with the maintenance of the project.
>>>>>>>>> >
>>>>>>>>> > In order to improve the situation, we need to solve the lack of
>>>>>>>>> >
>>>>>>>>> > committers and the lack of contributors.
>>>>>>>>> >
>>>>>>>>> > On the lack of committers:
>>>>>>>>> >
>>>>>>>>> > 1. Ideally, there are some of the current Flink committers who
>>>>>>>>> have
>>>>>>>>> >
>>>>>>>>> > the bandwidth and can help with reviewing PRs and merging them.
>>>>>>>>> >
>>>>>>>>> > 2. If that's not an option, it could be a consideration that
>>>>>>>>> current
>>>>>>>>> >
>>>>>>>>> > committers only approve and review PRs, that are approved by
>>>>>>>>> those who are
>>>>>>>>> > willing to contribute to Statefun and if the CI passes
>>>>>>>>> >
>>>>>>>>> > On the lack of contributors:
>>>>>>>>> >
>>>>>>>>> > 3. Next to having this discussion on the Dev and User mailing
>>>>>>>>> list, we
>>>>>>>>> >
>>>>>>>>> > can also create a blog with a call for new contributors on the
>>>>>>>>> Flink
>>>>>>>>> > project website, send out some tweets on the Flink / Statefun
>>>>>>>>> twitter
>>>>>>>>> > accounts, post messages on Slack etc. In that message, we would
>>>>>>>>> inform how
>>>>>>>>> > those that are interested in contributing can start and where
>>>>>>>>> they could
>>>>>>>>> > reach out for more information.
>>>>>>>>> >
>>>>>>>>> > There's also option 4. where a group of interested people would
>>>>>>>>> split
>>>>>>>>> >
>>>>>>>>> > Statefun from the Flink project and make it a separate top level
>>>>>>>>> project
>>>>>>>>> > under the Apache Flink umbrella (similar as recently has
>>>>>>>>> happened with
>>>>>>>>> > Flink Table Store, which has become Apache Paimon).
>>>>>>>>> >
>>>>>>>>> > If we see no improvements in the coming period, we should
>>>>>>>>> consider
>>>>>>>>> >
>>>>>>>>> > sunsetting Statefun and communicate that clearly to the users.
>>>>>>>>> >
>>>>>>>>> > I'm looking forward to your thoughts.
>>>>>>>>> >
>>>>>>>>> > Best regards,
>>>>>>>>> >
>>>>>>>>> > Martijn
>>>>>>>>> >
>>>>>>>>> > [1]
>>>>>>>>> https://nightlies.apache.org/flink/flink-statefun-docs-master/ <
>>>>>>>>> >
>>>>>>>>> > https://nightlies.apache.org/flink/flink-statefun-docs-master/>
>>>>>>>>> >
>>>>>>>>> > [2] https://github.com/apache/flink-statefun <
>>>>>>>>> >
>>>>>>>>> > https://github.com/apache/flink-statefun>
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>>
>>>>>>>>

Re: [DISCUSS] Status of Statefun Project

[Galen Warren via user <us...@flink.apache.org>]

I created a pull request for this: [FLINK-31619] Upgrade Stateful Functions
to Flink 1.16.1 by galenwarren · Pull Request #331 · apache/flink-statefun
(github.com) <https://github.com/apache/flink-statefun/pull/331>.

JIRA is here: [FLINK-31619] Upgrade Stateful Functions to Flink 1.16.1 -
ASF JIRA (apache.org)
<https://issues.apache.org/jira/browse/FLINK-31619?filter=-1>.

Statefun references 1.16.2, despite the title -- that version has come out
since the issue was created.

I figured out how to run all the playground tests locally, but it took a
bit of reworking of the playground setup with respect to Docker;
specifically, the Docker contexts used to build the example functions
needed to be broadened (i.e. moved up the tree) so that, if needed, local
artifacts/code can be accessed from within the containers at build time.
Then I made the Docker compose.yml configurable through environment
variables to allow for them to run in either the original manner -- i.e.
pulling artifacts from public repos -- or in a "local" mode, where
artifacts are pulled from local builds.

This process is a cleaner if the playground is a subfolder of the
flink-statefun project rather than be its own repository
(flink-statefun-playground), because then all the relative paths between
the playground files and the build artifacts are fixed. So, I'd like to
propose to move the playground files, modified as described above, to
flink-statefun/playground and retire flink-statefun-playground. I can
submit separate PR s those changes if everyone is on board.

Also, should I plan to do the same upgrade to handle Flink 1.17.x? It
should be easy to do, especially while the 1.16.x upgrade is fresh on my
mind.

Thanks.


On Fri, Aug 11, 2023 at 6:40 PM Galen Warren <ga...@cvillewarrens.com>
wrote:

> I'm done with the code to make Statefun compatible with Flink 1.16, and
> all the tests (including e2e succeed). The required changes were pretty
> minimal.
>
> I'm running into a bit of a chicken/egg problem executing the tests in
> flink-statefun-playground
> <https://github.com/apache/flink-statefun-playground>, though. That
> project seems to assume that all the various Statefun artifacts are built
> and deployed to the various public repositories already. I've looked into
> trying to redirect references to local artifacts; however, that's also
> tricky since all the building occurs in Docker containers.
>
> Gordon, is there a trick to running the sample code in
> flink-statefun-playground against yet-unreleased code that I'm missing?
>
> Thanks.
>
> On Sat, Jun 24, 2023 at 12:40 PM Galen Warren <ga...@cvillewarrens.com>
> wrote:
>
>> Great -- thanks!
>>
>> I'm going to be out of town for about a week but I'll take a look at this
>> when I'm back.
>>
>> On Tue, Jun 20, 2023 at 8:46 AM Martijn Visser <mv...@confluent.io>
>> wrote:
>>
>>> Hi Galen,
>>>
>>> Yes, I'll be more than happy to help with Statefun releases.
>>>
>>> Best regards,
>>>
>>> Martijn
>>>
>>> On Tue, Jun 20, 2023 at 2:21 PM Galen Warren <ga...@cvillewarrens.com>
>>> wrote:
>>>
>>>> Thanks.
>>>>
>>>> Martijn, to answer your question, I'd need to do a small amount of work
>>>> to get a PR ready, but not much. Happy to do it if we're deciding to
>>>> restart Statefun releases -- are we?
>>>>
>>>> -- Galen
>>>>
>>>> On Sat, Jun 17, 2023 at 9:47 AM Tzu-Li (Gordon) Tai <
>>>> tzulitai@apache.org> wrote:
>>>>
>>>>> > Perhaps he could weigh in on whether the combination of automated
>>>>> tests plus those smoke tests should be sufficient for testing with new
>>>>> Flink versions
>>>>>
>>>>> What we usually did at the bare minimum for new StateFun releases was
>>>>> the following:
>>>>>
>>>>>    1. Build tests (including the smoke tests in the e2e module, which
>>>>>    covers important tests like exactly-once verification)
>>>>>    2. Updating the flink-statefun-playground repo and manually
>>>>>    running all language examples there.
>>>>>
>>>>> If upgrading Flink versions was the only change in the release, I'd
>>>>> probably say that this is sufficient.
>>>>>
>>>>> Best,
>>>>> Gordon
>>>>>
>>>>> On Thu, Jun 15, 2023 at 5:25 AM Martijn Visser <
>>>>> martijnvisser@apache.org> wrote:
>>>>>
>>>>>> Let me know if you have a PR for a Flink update :)
>>>>>>
>>>>>> On Thu, Jun 8, 2023 at 5:52 PM Galen Warren via user <
>>>>>> user@flink.apache.org> wrote:
>>>>>>
>>>>>>> Thanks Martijn.
>>>>>>>
>>>>>>> Personally, I'm already using a local fork of Statefun that is
>>>>>>> compatible with Flink 1.16.x, so I wouldn't have any need for a released
>>>>>>> version compatible with 1.15.x. I'd be happy to do the PRs to modify
>>>>>>> Statefun to work with new versions of Flink as they come along.
>>>>>>>
>>>>>>> As for testing, Statefun does have unit tests and Gordon also sent
>>>>>>> me instructions a while back for how to do some additional smoke tests
>>>>>>> which are pretty straightforward. Perhaps he could weigh in on whether the
>>>>>>> combination of automated tests plus those smoke tests should be sufficient
>>>>>>> for testing with new Flink versions (I believe the answer is yes).
>>>>>>>
>>>>>>> -- Galen
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Jun 8, 2023 at 8:01 AM Martijn Visser <
>>>>>>> martijnvisser@apache.org> wrote:
>>>>>>>
>>>>>>>> Hi all,
>>>>>>>>
>>>>>>>> Apologies for the late reply.
>>>>>>>>
>>>>>>>> I'm willing to help out with merging requests in Statefun to keep
>>>>>>>> them
>>>>>>>> compatible with new Flink releases and create new releases. I do
>>>>>>>> think that
>>>>>>>> validation of the functionality of these releases depends a lot on
>>>>>>>> those
>>>>>>>> who do these compatibility updates, with PMC members helping out
>>>>>>>> with the
>>>>>>>> formal process.
>>>>>>>>
>>>>>>>> > Why can't the Apache Software Foundation allow community members
>>>>>>>> to bring
>>>>>>>> it up to date?
>>>>>>>>
>>>>>>>> There's nothing preventing anyone from reviewing any of the current
>>>>>>>> PRs or
>>>>>>>> opening new ones. However, none of them are approved [1], so
>>>>>>>> there's also
>>>>>>>> nothing to merge.
>>>>>>>>
>>>>>>>> > I believe that there are people and companies on this mailing list
>>>>>>>> interested in supporting Apache Flink Stateful Functions.
>>>>>>>>
>>>>>>>> If so, then now is the time to show.
>>>>>>>>
>>>>>>>> Would there be a preference to create a release with Galen's merged
>>>>>>>> compatibility update to Flink 1.15.2, or do we want to skip that
>>>>>>>> and go
>>>>>>>> straight to a newer version?
>>>>>>>>
>>>>>>>> Best regards,
>>>>>>>>
>>>>>>>> Martijn
>>>>>>>>
>>>>>>>> [1]
>>>>>>>>
>>>>>>>> https://github.com/apache/flink-statefun/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved
>>>>>>>>
>>>>>>>> On Tue, Jun 6, 2023 at 3:55 PM Marco Villalobos <
>>>>>>>> mvillalobos@kineteque.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> > Why can't the Apache Software Foundation allow community members
>>>>>>>> to bring
>>>>>>>> > it up to date?
>>>>>>>> >
>>>>>>>> > What's the process for that?
>>>>>>>> >
>>>>>>>> > I believe that there are people and companies on this mailing list
>>>>>>>> > interested in supporting Apache Flink Stateful Functions.
>>>>>>>> >
>>>>>>>> > You already had two people on this thread express interest.
>>>>>>>> >
>>>>>>>> > At the very least, we could keep the library versions up to date.
>>>>>>>> >
>>>>>>>> > There are only a small list of new features that might be
>>>>>>>> worthwhile:
>>>>>>>> >
>>>>>>>> > 1. event time processing
>>>>>>>> > 2. state rest api
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <ch...@apache.org>
>>>>>>>> wrote:
>>>>>>>> >
>>>>>>>> > If you were to fork it *and want to redistribute it* then the
>>>>>>>> short
>>>>>>>> > version is that
>>>>>>>> >
>>>>>>>> >    1. you have to adhere to the Apache licensing requirements
>>>>>>>> >    2. you have to make it clear that your fork does not belong to
>>>>>>>> the
>>>>>>>> >    Apache Flink project. (Trademarks and all that)
>>>>>>>> >
>>>>>>>> > Neither should be significant hurdles (there should also be
>>>>>>>> plenty of
>>>>>>>> > online resources regarding 1), and if you do this then you can
>>>>>>>> freely share
>>>>>>>> > your fork with others.
>>>>>>>> >
>>>>>>>> > I've also pinged Martijn to take a look at this thread.
>>>>>>>> > To my knowledge the project hasn't decided anything yet.
>>>>>>>> >
>>>>>>>> > On 27/05/2023 04:05, Galen Warren wrote:
>>>>>>>> >
>>>>>>>> > Ok, I get it. No interest.
>>>>>>>> >
>>>>>>>> > If this project is being abandoned, I guess I'll work with my own
>>>>>>>> fork. Is
>>>>>>>> > there anything I should consider here? Can I share it with other
>>>>>>>> people who
>>>>>>>> > use this project?
>>>>>>>> >
>>>>>>>> > On Tue, May 16, 2023 at 10:50 AM Galen Warren <
>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
>>>>>>>> > wrote:
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > Hi Martijn, since you opened this discussion thread, I'm curious
>>>>>>>> what your
>>>>>>>> > thoughts are in light of the responses? Thanks.
>>>>>>>> >
>>>>>>>> > On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <
>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
>>>>>>>> > wrote:
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > I use Apache Flink for stream processing, and StateFun as a
>>>>>>>> hand-off
>>>>>>>> >
>>>>>>>> > point for the rest of the application.
>>>>>>>> > It serves well as a bridge between a Flink Streaming job and
>>>>>>>> > micro-services.
>>>>>>>> >
>>>>>>>> > This is essentially how I use it as well, and I would also be sad
>>>>>>>> to see
>>>>>>>> > it sunsetted. It works well; I don't know that there is a lot of
>>>>>>>> new
>>>>>>>> > development required, but if there are no new Statefun releases,
>>>>>>>> then
>>>>>>>> > Statefun can only be used with older Flink versions.
>>>>>>>> >
>>>>>>>> > On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
>>>>>>>> mvillalobos@kineteque.com> wrote:
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > I am currently using Stateful Functions in my application.
>>>>>>>> >
>>>>>>>> > I use Apache Flink for stream processing, and StateFun as a
>>>>>>>> hand-off
>>>>>>>> > point for the rest of the application.
>>>>>>>> > It serves well as a bridge between a Flink Streaming job and
>>>>>>>> > micro-services.
>>>>>>>> >
>>>>>>>> > I would be disappointed if StateFun was sunsetted.  Its a good
>>>>>>>> idea.
>>>>>>>> >
>>>>>>>> > If there is anything I can do to help, as a contributor perhaps,
>>>>>>>> please
>>>>>>>> > let me know.
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <
>>>>>>>> martijnvisser@apache.org> <ma...@apache.org>
>>>>>>>> >
>>>>>>>> > wrote:
>>>>>>>> >
>>>>>>>> > Hi everyone,
>>>>>>>> >
>>>>>>>> > I want to open a discussion on the status of the Statefun Project
>>>>>>>> [1]
>>>>>>>> >
>>>>>>>> > in Apache Flink. As you might have noticed, there hasn't been much
>>>>>>>> > development over the past months in the Statefun repository [2].
>>>>>>>> There is
>>>>>>>> > currently a lack of active contributors and committers who are
>>>>>>>> able to help
>>>>>>>> > with the maintenance of the project.
>>>>>>>> >
>>>>>>>> > In order to improve the situation, we need to solve the lack of
>>>>>>>> >
>>>>>>>> > committers and the lack of contributors.
>>>>>>>> >
>>>>>>>> > On the lack of committers:
>>>>>>>> >
>>>>>>>> > 1. Ideally, there are some of the current Flink committers who
>>>>>>>> have
>>>>>>>> >
>>>>>>>> > the bandwidth and can help with reviewing PRs and merging them.
>>>>>>>> >
>>>>>>>> > 2. If that's not an option, it could be a consideration that
>>>>>>>> current
>>>>>>>> >
>>>>>>>> > committers only approve and review PRs, that are approved by
>>>>>>>> those who are
>>>>>>>> > willing to contribute to Statefun and if the CI passes
>>>>>>>> >
>>>>>>>> > On the lack of contributors:
>>>>>>>> >
>>>>>>>> > 3. Next to having this discussion on the Dev and User mailing
>>>>>>>> list, we
>>>>>>>> >
>>>>>>>> > can also create a blog with a call for new contributors on the
>>>>>>>> Flink
>>>>>>>> > project website, send out some tweets on the Flink / Statefun
>>>>>>>> twitter
>>>>>>>> > accounts, post messages on Slack etc. In that message, we would
>>>>>>>> inform how
>>>>>>>> > those that are interested in contributing can start and where
>>>>>>>> they could
>>>>>>>> > reach out for more information.
>>>>>>>> >
>>>>>>>> > There's also option 4. where a group of interested people would
>>>>>>>> split
>>>>>>>> >
>>>>>>>> > Statefun from the Flink project and make it a separate top level
>>>>>>>> project
>>>>>>>> > under the Apache Flink umbrella (similar as recently has happened
>>>>>>>> with
>>>>>>>> > Flink Table Store, which has become Apache Paimon).
>>>>>>>> >
>>>>>>>> > If we see no improvements in the coming period, we should consider
>>>>>>>> >
>>>>>>>> > sunsetting Statefun and communicate that clearly to the users.
>>>>>>>> >
>>>>>>>> > I'm looking forward to your thoughts.
>>>>>>>> >
>>>>>>>> > Best regards,
>>>>>>>> >
>>>>>>>> > Martijn
>>>>>>>> >
>>>>>>>> > [1]
>>>>>>>> https://nightlies.apache.org/flink/flink-statefun-docs-master/ <
>>>>>>>> >
>>>>>>>> > https://nightlies.apache.org/flink/flink-statefun-docs-master/>
>>>>>>>> >
>>>>>>>> > [2] https://github.com/apache/flink-statefun <
>>>>>>>> >
>>>>>>>> > https://github.com/apache/flink-statefun>
>>>>>>>> >
>>>>>>>> >
>>>>>>>> >
>>>>>>>> >
>>>>>>>> >
>>>>>>>>
>>>>>>>

Re: [DISCUSS] Status of Statefun Project

[Galen Warren <ga...@cvillewarrens.com.INVALID>]

I created a pull request for this: [FLINK-31619] Upgrade Stateful Functions
to Flink 1.16.1 by galenwarren · Pull Request #331 · apache/flink-statefun
(github.com) <https://github.com/apache/flink-statefun/pull/331>.

JIRA is here: [FLINK-31619] Upgrade Stateful Functions to Flink 1.16.1 -
ASF JIRA (apache.org)
<https://issues.apache.org/jira/browse/FLINK-31619?filter=-1>.

Statefun references 1.16.2, despite the title -- that version has come out
since the issue was created.

I figured out how to run all the playground tests locally, but it took a
bit of reworking of the playground setup with respect to Docker;
specifically, the Docker contexts used to build the example functions
needed to be broadened (i.e. moved up the tree) so that, if needed, local
artifacts/code can be accessed from within the containers at build time.
Then I made the Docker compose.yml configurable through environment
variables to allow for them to run in either the original manner -- i.e.
pulling artifacts from public repos -- or in a "local" mode, where
artifacts are pulled from local builds.

This process is a cleaner if the playground is a subfolder of the
flink-statefun project rather than be its own repository
(flink-statefun-playground), because then all the relative paths between
the playground files and the build artifacts are fixed. So, I'd like to
propose to move the playground files, modified as described above, to
flink-statefun/playground and retire flink-statefun-playground. I can
submit separate PR s those changes if everyone is on board.

Also, should I plan to do the same upgrade to handle Flink 1.17.x? It
should be easy to do, especially while the 1.16.x upgrade is fresh on my
mind.

Thanks.


On Fri, Aug 11, 2023 at 6:40 PM Galen Warren <ga...@cvillewarrens.com>
wrote:

> I'm done with the code to make Statefun compatible with Flink 1.16, and
> all the tests (including e2e succeed). The required changes were pretty
> minimal.
>
> I'm running into a bit of a chicken/egg problem executing the tests in
> flink-statefun-playground
> <https://github.com/apache/flink-statefun-playground>, though. That
> project seems to assume that all the various Statefun artifacts are built
> and deployed to the various public repositories already. I've looked into
> trying to redirect references to local artifacts; however, that's also
> tricky since all the building occurs in Docker containers.
>
> Gordon, is there a trick to running the sample code in
> flink-statefun-playground against yet-unreleased code that I'm missing?
>
> Thanks.
>
> On Sat, Jun 24, 2023 at 12:40 PM Galen Warren <ga...@cvillewarrens.com>
> wrote:
>
>> Great -- thanks!
>>
>> I'm going to be out of town for about a week but I'll take a look at this
>> when I'm back.
>>
>> On Tue, Jun 20, 2023 at 8:46 AM Martijn Visser <mv...@confluent.io>
>> wrote:
>>
>>> Hi Galen,
>>>
>>> Yes, I'll be more than happy to help with Statefun releases.
>>>
>>> Best regards,
>>>
>>> Martijn
>>>
>>> On Tue, Jun 20, 2023 at 2:21 PM Galen Warren <ga...@cvillewarrens.com>
>>> wrote:
>>>
>>>> Thanks.
>>>>
>>>> Martijn, to answer your question, I'd need to do a small amount of work
>>>> to get a PR ready, but not much. Happy to do it if we're deciding to
>>>> restart Statefun releases -- are we?
>>>>
>>>> -- Galen
>>>>
>>>> On Sat, Jun 17, 2023 at 9:47 AM Tzu-Li (Gordon) Tai <
>>>> tzulitai@apache.org> wrote:
>>>>
>>>>> > Perhaps he could weigh in on whether the combination of automated
>>>>> tests plus those smoke tests should be sufficient for testing with new
>>>>> Flink versions
>>>>>
>>>>> What we usually did at the bare minimum for new StateFun releases was
>>>>> the following:
>>>>>
>>>>>    1. Build tests (including the smoke tests in the e2e module, which
>>>>>    covers important tests like exactly-once verification)
>>>>>    2. Updating the flink-statefun-playground repo and manually
>>>>>    running all language examples there.
>>>>>
>>>>> If upgrading Flink versions was the only change in the release, I'd
>>>>> probably say that this is sufficient.
>>>>>
>>>>> Best,
>>>>> Gordon
>>>>>
>>>>> On Thu, Jun 15, 2023 at 5:25 AM Martijn Visser <
>>>>> martijnvisser@apache.org> wrote:
>>>>>
>>>>>> Let me know if you have a PR for a Flink update :)
>>>>>>
>>>>>> On Thu, Jun 8, 2023 at 5:52 PM Galen Warren via user <
>>>>>> user@flink.apache.org> wrote:
>>>>>>
>>>>>>> Thanks Martijn.
>>>>>>>
>>>>>>> Personally, I'm already using a local fork of Statefun that is
>>>>>>> compatible with Flink 1.16.x, so I wouldn't have any need for a released
>>>>>>> version compatible with 1.15.x. I'd be happy to do the PRs to modify
>>>>>>> Statefun to work with new versions of Flink as they come along.
>>>>>>>
>>>>>>> As for testing, Statefun does have unit tests and Gordon also sent
>>>>>>> me instructions a while back for how to do some additional smoke tests
>>>>>>> which are pretty straightforward. Perhaps he could weigh in on whether the
>>>>>>> combination of automated tests plus those smoke tests should be sufficient
>>>>>>> for testing with new Flink versions (I believe the answer is yes).
>>>>>>>
>>>>>>> -- Galen
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Jun 8, 2023 at 8:01 AM Martijn Visser <
>>>>>>> martijnvisser@apache.org> wrote:
>>>>>>>
>>>>>>>> Hi all,
>>>>>>>>
>>>>>>>> Apologies for the late reply.
>>>>>>>>
>>>>>>>> I'm willing to help out with merging requests in Statefun to keep
>>>>>>>> them
>>>>>>>> compatible with new Flink releases and create new releases. I do
>>>>>>>> think that
>>>>>>>> validation of the functionality of these releases depends a lot on
>>>>>>>> those
>>>>>>>> who do these compatibility updates, with PMC members helping out
>>>>>>>> with the
>>>>>>>> formal process.
>>>>>>>>
>>>>>>>> > Why can't the Apache Software Foundation allow community members
>>>>>>>> to bring
>>>>>>>> it up to date?
>>>>>>>>
>>>>>>>> There's nothing preventing anyone from reviewing any of the current
>>>>>>>> PRs or
>>>>>>>> opening new ones. However, none of them are approved [1], so
>>>>>>>> there's also
>>>>>>>> nothing to merge.
>>>>>>>>
>>>>>>>> > I believe that there are people and companies on this mailing list
>>>>>>>> interested in supporting Apache Flink Stateful Functions.
>>>>>>>>
>>>>>>>> If so, then now is the time to show.
>>>>>>>>
>>>>>>>> Would there be a preference to create a release with Galen's merged
>>>>>>>> compatibility update to Flink 1.15.2, or do we want to skip that
>>>>>>>> and go
>>>>>>>> straight to a newer version?
>>>>>>>>
>>>>>>>> Best regards,
>>>>>>>>
>>>>>>>> Martijn
>>>>>>>>
>>>>>>>> [1]
>>>>>>>>
>>>>>>>> https://github.com/apache/flink-statefun/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved
>>>>>>>>
>>>>>>>> On Tue, Jun 6, 2023 at 3:55 PM Marco Villalobos <
>>>>>>>> mvillalobos@kineteque.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> > Why can't the Apache Software Foundation allow community members
>>>>>>>> to bring
>>>>>>>> > it up to date?
>>>>>>>> >
>>>>>>>> > What's the process for that?
>>>>>>>> >
>>>>>>>> > I believe that there are people and companies on this mailing list
>>>>>>>> > interested in supporting Apache Flink Stateful Functions.
>>>>>>>> >
>>>>>>>> > You already had two people on this thread express interest.
>>>>>>>> >
>>>>>>>> > At the very least, we could keep the library versions up to date.
>>>>>>>> >
>>>>>>>> > There are only a small list of new features that might be
>>>>>>>> worthwhile:
>>>>>>>> >
>>>>>>>> > 1. event time processing
>>>>>>>> > 2. state rest api
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <ch...@apache.org>
>>>>>>>> wrote:
>>>>>>>> >
>>>>>>>> > If you were to fork it *and want to redistribute it* then the
>>>>>>>> short
>>>>>>>> > version is that
>>>>>>>> >
>>>>>>>> >    1. you have to adhere to the Apache licensing requirements
>>>>>>>> >    2. you have to make it clear that your fork does not belong to
>>>>>>>> the
>>>>>>>> >    Apache Flink project. (Trademarks and all that)
>>>>>>>> >
>>>>>>>> > Neither should be significant hurdles (there should also be
>>>>>>>> plenty of
>>>>>>>> > online resources regarding 1), and if you do this then you can
>>>>>>>> freely share
>>>>>>>> > your fork with others.
>>>>>>>> >
>>>>>>>> > I've also pinged Martijn to take a look at this thread.
>>>>>>>> > To my knowledge the project hasn't decided anything yet.
>>>>>>>> >
>>>>>>>> > On 27/05/2023 04:05, Galen Warren wrote:
>>>>>>>> >
>>>>>>>> > Ok, I get it. No interest.
>>>>>>>> >
>>>>>>>> > If this project is being abandoned, I guess I'll work with my own
>>>>>>>> fork. Is
>>>>>>>> > there anything I should consider here? Can I share it with other
>>>>>>>> people who
>>>>>>>> > use this project?
>>>>>>>> >
>>>>>>>> > On Tue, May 16, 2023 at 10:50 AM Galen Warren <
>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
>>>>>>>> > wrote:
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > Hi Martijn, since you opened this discussion thread, I'm curious
>>>>>>>> what your
>>>>>>>> > thoughts are in light of the responses? Thanks.
>>>>>>>> >
>>>>>>>> > On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <
>>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
>>>>>>>> > wrote:
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > I use Apache Flink for stream processing, and StateFun as a
>>>>>>>> hand-off
>>>>>>>> >
>>>>>>>> > point for the rest of the application.
>>>>>>>> > It serves well as a bridge between a Flink Streaming job and
>>>>>>>> > micro-services.
>>>>>>>> >
>>>>>>>> > This is essentially how I use it as well, and I would also be sad
>>>>>>>> to see
>>>>>>>> > it sunsetted. It works well; I don't know that there is a lot of
>>>>>>>> new
>>>>>>>> > development required, but if there are no new Statefun releases,
>>>>>>>> then
>>>>>>>> > Statefun can only be used with older Flink versions.
>>>>>>>> >
>>>>>>>> > On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
>>>>>>>> mvillalobos@kineteque.com> wrote:
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > I am currently using Stateful Functions in my application.
>>>>>>>> >
>>>>>>>> > I use Apache Flink for stream processing, and StateFun as a
>>>>>>>> hand-off
>>>>>>>> > point for the rest of the application.
>>>>>>>> > It serves well as a bridge between a Flink Streaming job and
>>>>>>>> > micro-services.
>>>>>>>> >
>>>>>>>> > I would be disappointed if StateFun was sunsetted.  Its a good
>>>>>>>> idea.
>>>>>>>> >
>>>>>>>> > If there is anything I can do to help, as a contributor perhaps,
>>>>>>>> please
>>>>>>>> > let me know.
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <
>>>>>>>> martijnvisser@apache.org> <ma...@apache.org>
>>>>>>>> >
>>>>>>>> > wrote:
>>>>>>>> >
>>>>>>>> > Hi everyone,
>>>>>>>> >
>>>>>>>> > I want to open a discussion on the status of the Statefun Project
>>>>>>>> [1]
>>>>>>>> >
>>>>>>>> > in Apache Flink. As you might have noticed, there hasn't been much
>>>>>>>> > development over the past months in the Statefun repository [2].
>>>>>>>> There is
>>>>>>>> > currently a lack of active contributors and committers who are
>>>>>>>> able to help
>>>>>>>> > with the maintenance of the project.
>>>>>>>> >
>>>>>>>> > In order to improve the situation, we need to solve the lack of
>>>>>>>> >
>>>>>>>> > committers and the lack of contributors.
>>>>>>>> >
>>>>>>>> > On the lack of committers:
>>>>>>>> >
>>>>>>>> > 1. Ideally, there are some of the current Flink committers who
>>>>>>>> have
>>>>>>>> >
>>>>>>>> > the bandwidth and can help with reviewing PRs and merging them.
>>>>>>>> >
>>>>>>>> > 2. If that's not an option, it could be a consideration that
>>>>>>>> current
>>>>>>>> >
>>>>>>>> > committers only approve and review PRs, that are approved by
>>>>>>>> those who are
>>>>>>>> > willing to contribute to Statefun and if the CI passes
>>>>>>>> >
>>>>>>>> > On the lack of contributors:
>>>>>>>> >
>>>>>>>> > 3. Next to having this discussion on the Dev and User mailing
>>>>>>>> list, we
>>>>>>>> >
>>>>>>>> > can also create a blog with a call for new contributors on the
>>>>>>>> Flink
>>>>>>>> > project website, send out some tweets on the Flink / Statefun
>>>>>>>> twitter
>>>>>>>> > accounts, post messages on Slack etc. In that message, we would
>>>>>>>> inform how
>>>>>>>> > those that are interested in contributing can start and where
>>>>>>>> they could
>>>>>>>> > reach out for more information.
>>>>>>>> >
>>>>>>>> > There's also option 4. where a group of interested people would
>>>>>>>> split
>>>>>>>> >
>>>>>>>> > Statefun from the Flink project and make it a separate top level
>>>>>>>> project
>>>>>>>> > under the Apache Flink umbrella (similar as recently has happened
>>>>>>>> with
>>>>>>>> > Flink Table Store, which has become Apache Paimon).
>>>>>>>> >
>>>>>>>> > If we see no improvements in the coming period, we should consider
>>>>>>>> >
>>>>>>>> > sunsetting Statefun and communicate that clearly to the users.
>>>>>>>> >
>>>>>>>> > I'm looking forward to your thoughts.
>>>>>>>> >
>>>>>>>> > Best regards,
>>>>>>>> >
>>>>>>>> > Martijn
>>>>>>>> >
>>>>>>>> > [1]
>>>>>>>> https://nightlies.apache.org/flink/flink-statefun-docs-master/ <
>>>>>>>> >
>>>>>>>> > https://nightlies.apache.org/flink/flink-statefun-docs-master/>
>>>>>>>> >
>>>>>>>> > [2] https://github.com/apache/flink-statefun <
>>>>>>>> >
>>>>>>>> > https://github.com/apache/flink-statefun>
>>>>>>>> >
>>>>>>>> >
>>>>>>>> >
>>>>>>>> >
>>>>>>>> >
>>>>>>>>
>>>>>>>

Re: [DISCUSS] Status of Statefun Project

[Galen Warren <ga...@cvillewarrens.com.INVALID>]

I'm done with the code to make Statefun compatible with Flink 1.16, and all
the tests (including e2e succeed). The required changes were pretty minimal.

I'm running into a bit of a chicken/egg problem executing the tests in
flink-statefun-playground
<https://github.com/apache/flink-statefun-playground>, though. That project
seems to assume that all the various Statefun artifacts are built and
deployed to the various public repositories already. I've looked into
trying to redirect references to local artifacts; however, that's also
tricky since all the building occurs in Docker containers.

Gordon, is there a trick to running the sample code in
flink-statefun-playground against yet-unreleased code that I'm missing?

Thanks.

On Sat, Jun 24, 2023 at 12:40 PM Galen Warren <ga...@cvillewarrens.com>
wrote:

> Great -- thanks!
>
> I'm going to be out of town for about a week but I'll take a look at this
> when I'm back.
>
> On Tue, Jun 20, 2023 at 8:46 AM Martijn Visser <mv...@confluent.io>
> wrote:
>
>> Hi Galen,
>>
>> Yes, I'll be more than happy to help with Statefun releases.
>>
>> Best regards,
>>
>> Martijn
>>
>> On Tue, Jun 20, 2023 at 2:21 PM Galen Warren <ga...@cvillewarrens.com>
>> wrote:
>>
>>> Thanks.
>>>
>>> Martijn, to answer your question, I'd need to do a small amount of work
>>> to get a PR ready, but not much. Happy to do it if we're deciding to
>>> restart Statefun releases -- are we?
>>>
>>> -- Galen
>>>
>>> On Sat, Jun 17, 2023 at 9:47 AM Tzu-Li (Gordon) Tai <tz...@apache.org>
>>> wrote:
>>>
>>>> > Perhaps he could weigh in on whether the combination of automated
>>>> tests plus those smoke tests should be sufficient for testing with new
>>>> Flink versions
>>>>
>>>> What we usually did at the bare minimum for new StateFun releases was
>>>> the following:
>>>>
>>>>    1. Build tests (including the smoke tests in the e2e module, which
>>>>    covers important tests like exactly-once verification)
>>>>    2. Updating the flink-statefun-playground repo and manually running
>>>>    all language examples there.
>>>>
>>>> If upgrading Flink versions was the only change in the release, I'd
>>>> probably say that this is sufficient.
>>>>
>>>> Best,
>>>> Gordon
>>>>
>>>> On Thu, Jun 15, 2023 at 5:25 AM Martijn Visser <
>>>> martijnvisser@apache.org> wrote:
>>>>
>>>>> Let me know if you have a PR for a Flink update :)
>>>>>
>>>>> On Thu, Jun 8, 2023 at 5:52 PM Galen Warren via user <
>>>>> user@flink.apache.org> wrote:
>>>>>
>>>>>> Thanks Martijn.
>>>>>>
>>>>>> Personally, I'm already using a local fork of Statefun that is
>>>>>> compatible with Flink 1.16.x, so I wouldn't have any need for a released
>>>>>> version compatible with 1.15.x. I'd be happy to do the PRs to modify
>>>>>> Statefun to work with new versions of Flink as they come along.
>>>>>>
>>>>>> As for testing, Statefun does have unit tests and Gordon also sent me
>>>>>> instructions a while back for how to do some additional smoke tests which
>>>>>> are pretty straightforward. Perhaps he could weigh in on whether the
>>>>>> combination of automated tests plus those smoke tests should be sufficient
>>>>>> for testing with new Flink versions (I believe the answer is yes).
>>>>>>
>>>>>> -- Galen
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Jun 8, 2023 at 8:01 AM Martijn Visser <
>>>>>> martijnvisser@apache.org> wrote:
>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> Apologies for the late reply.
>>>>>>>
>>>>>>> I'm willing to help out with merging requests in Statefun to keep
>>>>>>> them
>>>>>>> compatible with new Flink releases and create new releases. I do
>>>>>>> think that
>>>>>>> validation of the functionality of these releases depends a lot on
>>>>>>> those
>>>>>>> who do these compatibility updates, with PMC members helping out
>>>>>>> with the
>>>>>>> formal process.
>>>>>>>
>>>>>>> > Why can't the Apache Software Foundation allow community members
>>>>>>> to bring
>>>>>>> it up to date?
>>>>>>>
>>>>>>> There's nothing preventing anyone from reviewing any of the current
>>>>>>> PRs or
>>>>>>> opening new ones. However, none of them are approved [1], so there's
>>>>>>> also
>>>>>>> nothing to merge.
>>>>>>>
>>>>>>> > I believe that there are people and companies on this mailing list
>>>>>>> interested in supporting Apache Flink Stateful Functions.
>>>>>>>
>>>>>>> If so, then now is the time to show.
>>>>>>>
>>>>>>> Would there be a preference to create a release with Galen's merged
>>>>>>> compatibility update to Flink 1.15.2, or do we want to skip that and
>>>>>>> go
>>>>>>> straight to a newer version?
>>>>>>>
>>>>>>> Best regards,
>>>>>>>
>>>>>>> Martijn
>>>>>>>
>>>>>>> [1]
>>>>>>>
>>>>>>> https://github.com/apache/flink-statefun/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved
>>>>>>>
>>>>>>> On Tue, Jun 6, 2023 at 3:55 PM Marco Villalobos <
>>>>>>> mvillalobos@kineteque.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> > Why can't the Apache Software Foundation allow community members
>>>>>>> to bring
>>>>>>> > it up to date?
>>>>>>> >
>>>>>>> > What's the process for that?
>>>>>>> >
>>>>>>> > I believe that there are people and companies on this mailing list
>>>>>>> > interested in supporting Apache Flink Stateful Functions.
>>>>>>> >
>>>>>>> > You already had two people on this thread express interest.
>>>>>>> >
>>>>>>> > At the very least, we could keep the library versions up to date.
>>>>>>> >
>>>>>>> > There are only a small list of new features that might be
>>>>>>> worthwhile:
>>>>>>> >
>>>>>>> > 1. event time processing
>>>>>>> > 2. state rest api
>>>>>>> >
>>>>>>> >
>>>>>>> > On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <ch...@apache.org>
>>>>>>> wrote:
>>>>>>> >
>>>>>>> > If you were to fork it *and want to redistribute it* then the short
>>>>>>> > version is that
>>>>>>> >
>>>>>>> >    1. you have to adhere to the Apache licensing requirements
>>>>>>> >    2. you have to make it clear that your fork does not belong to
>>>>>>> the
>>>>>>> >    Apache Flink project. (Trademarks and all that)
>>>>>>> >
>>>>>>> > Neither should be significant hurdles (there should also be plenty
>>>>>>> of
>>>>>>> > online resources regarding 1), and if you do this then you can
>>>>>>> freely share
>>>>>>> > your fork with others.
>>>>>>> >
>>>>>>> > I've also pinged Martijn to take a look at this thread.
>>>>>>> > To my knowledge the project hasn't decided anything yet.
>>>>>>> >
>>>>>>> > On 27/05/2023 04:05, Galen Warren wrote:
>>>>>>> >
>>>>>>> > Ok, I get it. No interest.
>>>>>>> >
>>>>>>> > If this project is being abandoned, I guess I'll work with my own
>>>>>>> fork. Is
>>>>>>> > there anything I should consider here? Can I share it with other
>>>>>>> people who
>>>>>>> > use this project?
>>>>>>> >
>>>>>>> > On Tue, May 16, 2023 at 10:50 AM Galen Warren <
>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
>>>>>>> > wrote:
>>>>>>> >
>>>>>>> >
>>>>>>> > Hi Martijn, since you opened this discussion thread, I'm curious
>>>>>>> what your
>>>>>>> > thoughts are in light of the responses? Thanks.
>>>>>>> >
>>>>>>> > On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <
>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
>>>>>>> > wrote:
>>>>>>> >
>>>>>>> >
>>>>>>> > I use Apache Flink for stream processing, and StateFun as a
>>>>>>> hand-off
>>>>>>> >
>>>>>>> > point for the rest of the application.
>>>>>>> > It serves well as a bridge between a Flink Streaming job and
>>>>>>> > micro-services.
>>>>>>> >
>>>>>>> > This is essentially how I use it as well, and I would also be sad
>>>>>>> to see
>>>>>>> > it sunsetted. It works well; I don't know that there is a lot of
>>>>>>> new
>>>>>>> > development required, but if there are no new Statefun releases,
>>>>>>> then
>>>>>>> > Statefun can only be used with older Flink versions.
>>>>>>> >
>>>>>>> > On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
>>>>>>> mvillalobos@kineteque.com> wrote:
>>>>>>> >
>>>>>>> >
>>>>>>> > I am currently using Stateful Functions in my application.
>>>>>>> >
>>>>>>> > I use Apache Flink for stream processing, and StateFun as a
>>>>>>> hand-off
>>>>>>> > point for the rest of the application.
>>>>>>> > It serves well as a bridge between a Flink Streaming job and
>>>>>>> > micro-services.
>>>>>>> >
>>>>>>> > I would be disappointed if StateFun was sunsetted.  Its a good
>>>>>>> idea.
>>>>>>> >
>>>>>>> > If there is anything I can do to help, as a contributor perhaps,
>>>>>>> please
>>>>>>> > let me know.
>>>>>>> >
>>>>>>> >
>>>>>>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <
>>>>>>> martijnvisser@apache.org> <ma...@apache.org>
>>>>>>> >
>>>>>>> > wrote:
>>>>>>> >
>>>>>>> > Hi everyone,
>>>>>>> >
>>>>>>> > I want to open a discussion on the status of the Statefun Project
>>>>>>> [1]
>>>>>>> >
>>>>>>> > in Apache Flink. As you might have noticed, there hasn't been much
>>>>>>> > development over the past months in the Statefun repository [2].
>>>>>>> There is
>>>>>>> > currently a lack of active contributors and committers who are
>>>>>>> able to help
>>>>>>> > with the maintenance of the project.
>>>>>>> >
>>>>>>> > In order to improve the situation, we need to solve the lack of
>>>>>>> >
>>>>>>> > committers and the lack of contributors.
>>>>>>> >
>>>>>>> > On the lack of committers:
>>>>>>> >
>>>>>>> > 1. Ideally, there are some of the current Flink committers who have
>>>>>>> >
>>>>>>> > the bandwidth and can help with reviewing PRs and merging them.
>>>>>>> >
>>>>>>> > 2. If that's not an option, it could be a consideration that
>>>>>>> current
>>>>>>> >
>>>>>>> > committers only approve and review PRs, that are approved by those
>>>>>>> who are
>>>>>>> > willing to contribute to Statefun and if the CI passes
>>>>>>> >
>>>>>>> > On the lack of contributors:
>>>>>>> >
>>>>>>> > 3. Next to having this discussion on the Dev and User mailing
>>>>>>> list, we
>>>>>>> >
>>>>>>> > can also create a blog with a call for new contributors on the
>>>>>>> Flink
>>>>>>> > project website, send out some tweets on the Flink / Statefun
>>>>>>> twitter
>>>>>>> > accounts, post messages on Slack etc. In that message, we would
>>>>>>> inform how
>>>>>>> > those that are interested in contributing can start and where they
>>>>>>> could
>>>>>>> > reach out for more information.
>>>>>>> >
>>>>>>> > There's also option 4. where a group of interested people would
>>>>>>> split
>>>>>>> >
>>>>>>> > Statefun from the Flink project and make it a separate top level
>>>>>>> project
>>>>>>> > under the Apache Flink umbrella (similar as recently has happened
>>>>>>> with
>>>>>>> > Flink Table Store, which has become Apache Paimon).
>>>>>>> >
>>>>>>> > If we see no improvements in the coming period, we should consider
>>>>>>> >
>>>>>>> > sunsetting Statefun and communicate that clearly to the users.
>>>>>>> >
>>>>>>> > I'm looking forward to your thoughts.
>>>>>>> >
>>>>>>> > Best regards,
>>>>>>> >
>>>>>>> > Martijn
>>>>>>> >
>>>>>>> > [1] https://nightlies.apache.org/flink/flink-statefun-docs-master/
>>>>>>> <
>>>>>>> >
>>>>>>> > https://nightlies.apache.org/flink/flink-statefun-docs-master/>
>>>>>>> >
>>>>>>> > [2] https://github.com/apache/flink-statefun <
>>>>>>> >
>>>>>>> > https://github.com/apache/flink-statefun>
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>>
>>>>>>

Re: [DISCUSS] Status of Statefun Project

[Filip Karnicki <fi...@gmail.com>]

This is great news, as we're using statefun as well.

Please don't hesitate to let me know if you need me to do some additional
testing on a real life prod-like setup.

On Sat, 24 Jun 2023 at 18:41, Galen Warren via user <us...@flink.apache.org>
wrote:

> Great -- thanks!
>
> I'm going to be out of town for about a week but I'll take a look at this
> when I'm back.
>
> On Tue, Jun 20, 2023 at 8:46 AM Martijn Visser <mv...@confluent.io>
> wrote:
>
>> Hi Galen,
>>
>> Yes, I'll be more than happy to help with Statefun releases.
>>
>> Best regards,
>>
>> Martijn
>>
>> On Tue, Jun 20, 2023 at 2:21 PM Galen Warren <ga...@cvillewarrens.com>
>> wrote:
>>
>>> Thanks.
>>>
>>> Martijn, to answer your question, I'd need to do a small amount of work
>>> to get a PR ready, but not much. Happy to do it if we're deciding to
>>> restart Statefun releases -- are we?
>>>
>>> -- Galen
>>>
>>> On Sat, Jun 17, 2023 at 9:47 AM Tzu-Li (Gordon) Tai <tz...@apache.org>
>>> wrote:
>>>
>>>> > Perhaps he could weigh in on whether the combination of automated
>>>> tests plus those smoke tests should be sufficient for testing with new
>>>> Flink versions
>>>>
>>>> What we usually did at the bare minimum for new StateFun releases was
>>>> the following:
>>>>
>>>>    1. Build tests (including the smoke tests in the e2e module, which
>>>>    covers important tests like exactly-once verification)
>>>>    2. Updating the flink-statefun-playground repo and manually running
>>>>    all language examples there.
>>>>
>>>> If upgrading Flink versions was the only change in the release, I'd
>>>> probably say that this is sufficient.
>>>>
>>>> Best,
>>>> Gordon
>>>>
>>>> On Thu, Jun 15, 2023 at 5:25 AM Martijn Visser <
>>>> martijnvisser@apache.org> wrote:
>>>>
>>>>> Let me know if you have a PR for a Flink update :)
>>>>>
>>>>> On Thu, Jun 8, 2023 at 5:52 PM Galen Warren via user <
>>>>> user@flink.apache.org> wrote:
>>>>>
>>>>>> Thanks Martijn.
>>>>>>
>>>>>> Personally, I'm already using a local fork of Statefun that is
>>>>>> compatible with Flink 1.16.x, so I wouldn't have any need for a released
>>>>>> version compatible with 1.15.x. I'd be happy to do the PRs to modify
>>>>>> Statefun to work with new versions of Flink as they come along.
>>>>>>
>>>>>> As for testing, Statefun does have unit tests and Gordon also sent me
>>>>>> instructions a while back for how to do some additional smoke tests which
>>>>>> are pretty straightforward. Perhaps he could weigh in on whether the
>>>>>> combination of automated tests plus those smoke tests should be sufficient
>>>>>> for testing with new Flink versions (I believe the answer is yes).
>>>>>>
>>>>>> -- Galen
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Jun 8, 2023 at 8:01 AM Martijn Visser <
>>>>>> martijnvisser@apache.org> wrote:
>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> Apologies for the late reply.
>>>>>>>
>>>>>>> I'm willing to help out with merging requests in Statefun to keep
>>>>>>> them
>>>>>>> compatible with new Flink releases and create new releases. I do
>>>>>>> think that
>>>>>>> validation of the functionality of these releases depends a lot on
>>>>>>> those
>>>>>>> who do these compatibility updates, with PMC members helping out
>>>>>>> with the
>>>>>>> formal process.
>>>>>>>
>>>>>>> > Why can't the Apache Software Foundation allow community members
>>>>>>> to bring
>>>>>>> it up to date?
>>>>>>>
>>>>>>> There's nothing preventing anyone from reviewing any of the current
>>>>>>> PRs or
>>>>>>> opening new ones. However, none of them are approved [1], so there's
>>>>>>> also
>>>>>>> nothing to merge.
>>>>>>>
>>>>>>> > I believe that there are people and companies on this mailing list
>>>>>>> interested in supporting Apache Flink Stateful Functions.
>>>>>>>
>>>>>>> If so, then now is the time to show.
>>>>>>>
>>>>>>> Would there be a preference to create a release with Galen's merged
>>>>>>> compatibility update to Flink 1.15.2, or do we want to skip that and
>>>>>>> go
>>>>>>> straight to a newer version?
>>>>>>>
>>>>>>> Best regards,
>>>>>>>
>>>>>>> Martijn
>>>>>>>
>>>>>>> [1]
>>>>>>>
>>>>>>> https://github.com/apache/flink-statefun/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved
>>>>>>>
>>>>>>> On Tue, Jun 6, 2023 at 3:55 PM Marco Villalobos <
>>>>>>> mvillalobos@kineteque.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> > Why can't the Apache Software Foundation allow community members
>>>>>>> to bring
>>>>>>> > it up to date?
>>>>>>> >
>>>>>>> > What's the process for that?
>>>>>>> >
>>>>>>> > I believe that there are people and companies on this mailing list
>>>>>>> > interested in supporting Apache Flink Stateful Functions.
>>>>>>> >
>>>>>>> > You already had two people on this thread express interest.
>>>>>>> >
>>>>>>> > At the very least, we could keep the library versions up to date.
>>>>>>> >
>>>>>>> > There are only a small list of new features that might be
>>>>>>> worthwhile:
>>>>>>> >
>>>>>>> > 1. event time processing
>>>>>>> > 2. state rest api
>>>>>>> >
>>>>>>> >
>>>>>>> > On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <ch...@apache.org>
>>>>>>> wrote:
>>>>>>> >
>>>>>>> > If you were to fork it *and want to redistribute it* then the short
>>>>>>> > version is that
>>>>>>> >
>>>>>>> >    1. you have to adhere to the Apache licensing requirements
>>>>>>> >    2. you have to make it clear that your fork does not belong to
>>>>>>> the
>>>>>>> >    Apache Flink project. (Trademarks and all that)
>>>>>>> >
>>>>>>> > Neither should be significant hurdles (there should also be plenty
>>>>>>> of
>>>>>>> > online resources regarding 1), and if you do this then you can
>>>>>>> freely share
>>>>>>> > your fork with others.
>>>>>>> >
>>>>>>> > I've also pinged Martijn to take a look at this thread.
>>>>>>> > To my knowledge the project hasn't decided anything yet.
>>>>>>> >
>>>>>>> > On 27/05/2023 04:05, Galen Warren wrote:
>>>>>>> >
>>>>>>> > Ok, I get it. No interest.
>>>>>>> >
>>>>>>> > If this project is being abandoned, I guess I'll work with my own
>>>>>>> fork. Is
>>>>>>> > there anything I should consider here? Can I share it with other
>>>>>>> people who
>>>>>>> > use this project?
>>>>>>> >
>>>>>>> > On Tue, May 16, 2023 at 10:50 AM Galen Warren <
>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
>>>>>>> > wrote:
>>>>>>> >
>>>>>>> >
>>>>>>> > Hi Martijn, since you opened this discussion thread, I'm curious
>>>>>>> what your
>>>>>>> > thoughts are in light of the responses? Thanks.
>>>>>>> >
>>>>>>> > On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <
>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
>>>>>>> > wrote:
>>>>>>> >
>>>>>>> >
>>>>>>> > I use Apache Flink for stream processing, and StateFun as a
>>>>>>> hand-off
>>>>>>> >
>>>>>>> > point for the rest of the application.
>>>>>>> > It serves well as a bridge between a Flink Streaming job and
>>>>>>> > micro-services.
>>>>>>> >
>>>>>>> > This is essentially how I use it as well, and I would also be sad
>>>>>>> to see
>>>>>>> > it sunsetted. It works well; I don't know that there is a lot of
>>>>>>> new
>>>>>>> > development required, but if there are no new Statefun releases,
>>>>>>> then
>>>>>>> > Statefun can only be used with older Flink versions.
>>>>>>> >
>>>>>>> > On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
>>>>>>> mvillalobos@kineteque.com> wrote:
>>>>>>> >
>>>>>>> >
>>>>>>> > I am currently using Stateful Functions in my application.
>>>>>>> >
>>>>>>> > I use Apache Flink for stream processing, and StateFun as a
>>>>>>> hand-off
>>>>>>> > point for the rest of the application.
>>>>>>> > It serves well as a bridge between a Flink Streaming job and
>>>>>>> > micro-services.
>>>>>>> >
>>>>>>> > I would be disappointed if StateFun was sunsetted.  Its a good
>>>>>>> idea.
>>>>>>> >
>>>>>>> > If there is anything I can do to help, as a contributor perhaps,
>>>>>>> please
>>>>>>> > let me know.
>>>>>>> >
>>>>>>> >
>>>>>>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <
>>>>>>> martijnvisser@apache.org> <ma...@apache.org>
>>>>>>> >
>>>>>>> > wrote:
>>>>>>> >
>>>>>>> > Hi everyone,
>>>>>>> >
>>>>>>> > I want to open a discussion on the status of the Statefun Project
>>>>>>> [1]
>>>>>>> >
>>>>>>> > in Apache Flink. As you might have noticed, there hasn't been much
>>>>>>> > development over the past months in the Statefun repository [2].
>>>>>>> There is
>>>>>>> > currently a lack of active contributors and committers who are
>>>>>>> able to help
>>>>>>> > with the maintenance of the project.
>>>>>>> >
>>>>>>> > In order to improve the situation, we need to solve the lack of
>>>>>>> >
>>>>>>> > committers and the lack of contributors.
>>>>>>> >
>>>>>>> > On the lack of committers:
>>>>>>> >
>>>>>>> > 1. Ideally, there are some of the current Flink committers who have
>>>>>>> >
>>>>>>> > the bandwidth and can help with reviewing PRs and merging them.
>>>>>>> >
>>>>>>> > 2. If that's not an option, it could be a consideration that
>>>>>>> current
>>>>>>> >
>>>>>>> > committers only approve and review PRs, that are approved by those
>>>>>>> who are
>>>>>>> > willing to contribute to Statefun and if the CI passes
>>>>>>> >
>>>>>>> > On the lack of contributors:
>>>>>>> >
>>>>>>> > 3. Next to having this discussion on the Dev and User mailing
>>>>>>> list, we
>>>>>>> >
>>>>>>> > can also create a blog with a call for new contributors on the
>>>>>>> Flink
>>>>>>> > project website, send out some tweets on the Flink / Statefun
>>>>>>> twitter
>>>>>>> > accounts, post messages on Slack etc. In that message, we would
>>>>>>> inform how
>>>>>>> > those that are interested in contributing can start and where they
>>>>>>> could
>>>>>>> > reach out for more information.
>>>>>>> >
>>>>>>> > There's also option 4. where a group of interested people would
>>>>>>> split
>>>>>>> >
>>>>>>> > Statefun from the Flink project and make it a separate top level
>>>>>>> project
>>>>>>> > under the Apache Flink umbrella (similar as recently has happened
>>>>>>> with
>>>>>>> > Flink Table Store, which has become Apache Paimon).
>>>>>>> >
>>>>>>> > If we see no improvements in the coming period, we should consider
>>>>>>> >
>>>>>>> > sunsetting Statefun and communicate that clearly to the users.
>>>>>>> >
>>>>>>> > I'm looking forward to your thoughts.
>>>>>>> >
>>>>>>> > Best regards,
>>>>>>> >
>>>>>>> > Martijn
>>>>>>> >
>>>>>>> > [1] https://nightlies.apache.org/flink/flink-statefun-docs-master/
>>>>>>> <
>>>>>>> >
>>>>>>> > https://nightlies.apache.org/flink/flink-statefun-docs-master/>
>>>>>>> >
>>>>>>> > [2] https://github.com/apache/flink-statefun <
>>>>>>> >
>>>>>>> > https://github.com/apache/flink-statefun>
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>>
>>>>>>

Re: [DISCUSS] Status of Statefun Project

[Galen Warren via user <us...@flink.apache.org>]

I'm done with the code to make Statefun compatible with Flink 1.16, and all
the tests (including e2e succeed). The required changes were pretty minimal.

I'm running into a bit of a chicken/egg problem executing the tests in
flink-statefun-playground
<https://github.com/apache/flink-statefun-playground>, though. That project
seems to assume that all the various Statefun artifacts are built and
deployed to the various public repositories already. I've looked into
trying to redirect references to local artifacts; however, that's also
tricky since all the building occurs in Docker containers.

Gordon, is there a trick to running the sample code in
flink-statefun-playground against yet-unreleased code that I'm missing?

Thanks.

On Sat, Jun 24, 2023 at 12:40 PM Galen Warren <ga...@cvillewarrens.com>
wrote:

> Great -- thanks!
>
> I'm going to be out of town for about a week but I'll take a look at this
> when I'm back.
>
> On Tue, Jun 20, 2023 at 8:46 AM Martijn Visser <mv...@confluent.io>
> wrote:
>
>> Hi Galen,
>>
>> Yes, I'll be more than happy to help with Statefun releases.
>>
>> Best regards,
>>
>> Martijn
>>
>> On Tue, Jun 20, 2023 at 2:21 PM Galen Warren <ga...@cvillewarrens.com>
>> wrote:
>>
>>> Thanks.
>>>
>>> Martijn, to answer your question, I'd need to do a small amount of work
>>> to get a PR ready, but not much. Happy to do it if we're deciding to
>>> restart Statefun releases -- are we?
>>>
>>> -- Galen
>>>
>>> On Sat, Jun 17, 2023 at 9:47 AM Tzu-Li (Gordon) Tai <tz...@apache.org>
>>> wrote:
>>>
>>>> > Perhaps he could weigh in on whether the combination of automated
>>>> tests plus those smoke tests should be sufficient for testing with new
>>>> Flink versions
>>>>
>>>> What we usually did at the bare minimum for new StateFun releases was
>>>> the following:
>>>>
>>>>    1. Build tests (including the smoke tests in the e2e module, which
>>>>    covers important tests like exactly-once verification)
>>>>    2. Updating the flink-statefun-playground repo and manually running
>>>>    all language examples there.
>>>>
>>>> If upgrading Flink versions was the only change in the release, I'd
>>>> probably say that this is sufficient.
>>>>
>>>> Best,
>>>> Gordon
>>>>
>>>> On Thu, Jun 15, 2023 at 5:25 AM Martijn Visser <
>>>> martijnvisser@apache.org> wrote:
>>>>
>>>>> Let me know if you have a PR for a Flink update :)
>>>>>
>>>>> On Thu, Jun 8, 2023 at 5:52 PM Galen Warren via user <
>>>>> user@flink.apache.org> wrote:
>>>>>
>>>>>> Thanks Martijn.
>>>>>>
>>>>>> Personally, I'm already using a local fork of Statefun that is
>>>>>> compatible with Flink 1.16.x, so I wouldn't have any need for a released
>>>>>> version compatible with 1.15.x. I'd be happy to do the PRs to modify
>>>>>> Statefun to work with new versions of Flink as they come along.
>>>>>>
>>>>>> As for testing, Statefun does have unit tests and Gordon also sent me
>>>>>> instructions a while back for how to do some additional smoke tests which
>>>>>> are pretty straightforward. Perhaps he could weigh in on whether the
>>>>>> combination of automated tests plus those smoke tests should be sufficient
>>>>>> for testing with new Flink versions (I believe the answer is yes).
>>>>>>
>>>>>> -- Galen
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Jun 8, 2023 at 8:01 AM Martijn Visser <
>>>>>> martijnvisser@apache.org> wrote:
>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> Apologies for the late reply.
>>>>>>>
>>>>>>> I'm willing to help out with merging requests in Statefun to keep
>>>>>>> them
>>>>>>> compatible with new Flink releases and create new releases. I do
>>>>>>> think that
>>>>>>> validation of the functionality of these releases depends a lot on
>>>>>>> those
>>>>>>> who do these compatibility updates, with PMC members helping out
>>>>>>> with the
>>>>>>> formal process.
>>>>>>>
>>>>>>> > Why can't the Apache Software Foundation allow community members
>>>>>>> to bring
>>>>>>> it up to date?
>>>>>>>
>>>>>>> There's nothing preventing anyone from reviewing any of the current
>>>>>>> PRs or
>>>>>>> opening new ones. However, none of them are approved [1], so there's
>>>>>>> also
>>>>>>> nothing to merge.
>>>>>>>
>>>>>>> > I believe that there are people and companies on this mailing list
>>>>>>> interested in supporting Apache Flink Stateful Functions.
>>>>>>>
>>>>>>> If so, then now is the time to show.
>>>>>>>
>>>>>>> Would there be a preference to create a release with Galen's merged
>>>>>>> compatibility update to Flink 1.15.2, or do we want to skip that and
>>>>>>> go
>>>>>>> straight to a newer version?
>>>>>>>
>>>>>>> Best regards,
>>>>>>>
>>>>>>> Martijn
>>>>>>>
>>>>>>> [1]
>>>>>>>
>>>>>>> https://github.com/apache/flink-statefun/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved
>>>>>>>
>>>>>>> On Tue, Jun 6, 2023 at 3:55 PM Marco Villalobos <
>>>>>>> mvillalobos@kineteque.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> > Why can't the Apache Software Foundation allow community members
>>>>>>> to bring
>>>>>>> > it up to date?
>>>>>>> >
>>>>>>> > What's the process for that?
>>>>>>> >
>>>>>>> > I believe that there are people and companies on this mailing list
>>>>>>> > interested in supporting Apache Flink Stateful Functions.
>>>>>>> >
>>>>>>> > You already had two people on this thread express interest.
>>>>>>> >
>>>>>>> > At the very least, we could keep the library versions up to date.
>>>>>>> >
>>>>>>> > There are only a small list of new features that might be
>>>>>>> worthwhile:
>>>>>>> >
>>>>>>> > 1. event time processing
>>>>>>> > 2. state rest api
>>>>>>> >
>>>>>>> >
>>>>>>> > On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <ch...@apache.org>
>>>>>>> wrote:
>>>>>>> >
>>>>>>> > If you were to fork it *and want to redistribute it* then the short
>>>>>>> > version is that
>>>>>>> >
>>>>>>> >    1. you have to adhere to the Apache licensing requirements
>>>>>>> >    2. you have to make it clear that your fork does not belong to
>>>>>>> the
>>>>>>> >    Apache Flink project. (Trademarks and all that)
>>>>>>> >
>>>>>>> > Neither should be significant hurdles (there should also be plenty
>>>>>>> of
>>>>>>> > online resources regarding 1), and if you do this then you can
>>>>>>> freely share
>>>>>>> > your fork with others.
>>>>>>> >
>>>>>>> > I've also pinged Martijn to take a look at this thread.
>>>>>>> > To my knowledge the project hasn't decided anything yet.
>>>>>>> >
>>>>>>> > On 27/05/2023 04:05, Galen Warren wrote:
>>>>>>> >
>>>>>>> > Ok, I get it. No interest.
>>>>>>> >
>>>>>>> > If this project is being abandoned, I guess I'll work with my own
>>>>>>> fork. Is
>>>>>>> > there anything I should consider here? Can I share it with other
>>>>>>> people who
>>>>>>> > use this project?
>>>>>>> >
>>>>>>> > On Tue, May 16, 2023 at 10:50 AM Galen Warren <
>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
>>>>>>> > wrote:
>>>>>>> >
>>>>>>> >
>>>>>>> > Hi Martijn, since you opened this discussion thread, I'm curious
>>>>>>> what your
>>>>>>> > thoughts are in light of the responses? Thanks.
>>>>>>> >
>>>>>>> > On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <
>>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
>>>>>>> > wrote:
>>>>>>> >
>>>>>>> >
>>>>>>> > I use Apache Flink for stream processing, and StateFun as a
>>>>>>> hand-off
>>>>>>> >
>>>>>>> > point for the rest of the application.
>>>>>>> > It serves well as a bridge between a Flink Streaming job and
>>>>>>> > micro-services.
>>>>>>> >
>>>>>>> > This is essentially how I use it as well, and I would also be sad
>>>>>>> to see
>>>>>>> > it sunsetted. It works well; I don't know that there is a lot of
>>>>>>> new
>>>>>>> > development required, but if there are no new Statefun releases,
>>>>>>> then
>>>>>>> > Statefun can only be used with older Flink versions.
>>>>>>> >
>>>>>>> > On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
>>>>>>> mvillalobos@kineteque.com> wrote:
>>>>>>> >
>>>>>>> >
>>>>>>> > I am currently using Stateful Functions in my application.
>>>>>>> >
>>>>>>> > I use Apache Flink for stream processing, and StateFun as a
>>>>>>> hand-off
>>>>>>> > point for the rest of the application.
>>>>>>> > It serves well as a bridge between a Flink Streaming job and
>>>>>>> > micro-services.
>>>>>>> >
>>>>>>> > I would be disappointed if StateFun was sunsetted.  Its a good
>>>>>>> idea.
>>>>>>> >
>>>>>>> > If there is anything I can do to help, as a contributor perhaps,
>>>>>>> please
>>>>>>> > let me know.
>>>>>>> >
>>>>>>> >
>>>>>>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <
>>>>>>> martijnvisser@apache.org> <ma...@apache.org>
>>>>>>> >
>>>>>>> > wrote:
>>>>>>> >
>>>>>>> > Hi everyone,
>>>>>>> >
>>>>>>> > I want to open a discussion on the status of the Statefun Project
>>>>>>> [1]
>>>>>>> >
>>>>>>> > in Apache Flink. As you might have noticed, there hasn't been much
>>>>>>> > development over the past months in the Statefun repository [2].
>>>>>>> There is
>>>>>>> > currently a lack of active contributors and committers who are
>>>>>>> able to help
>>>>>>> > with the maintenance of the project.
>>>>>>> >
>>>>>>> > In order to improve the situation, we need to solve the lack of
>>>>>>> >
>>>>>>> > committers and the lack of contributors.
>>>>>>> >
>>>>>>> > On the lack of committers:
>>>>>>> >
>>>>>>> > 1. Ideally, there are some of the current Flink committers who have
>>>>>>> >
>>>>>>> > the bandwidth and can help with reviewing PRs and merging them.
>>>>>>> >
>>>>>>> > 2. If that's not an option, it could be a consideration that
>>>>>>> current
>>>>>>> >
>>>>>>> > committers only approve and review PRs, that are approved by those
>>>>>>> who are
>>>>>>> > willing to contribute to Statefun and if the CI passes
>>>>>>> >
>>>>>>> > On the lack of contributors:
>>>>>>> >
>>>>>>> > 3. Next to having this discussion on the Dev and User mailing
>>>>>>> list, we
>>>>>>> >
>>>>>>> > can also create a blog with a call for new contributors on the
>>>>>>> Flink
>>>>>>> > project website, send out some tweets on the Flink / Statefun
>>>>>>> twitter
>>>>>>> > accounts, post messages on Slack etc. In that message, we would
>>>>>>> inform how
>>>>>>> > those that are interested in contributing can start and where they
>>>>>>> could
>>>>>>> > reach out for more information.
>>>>>>> >
>>>>>>> > There's also option 4. where a group of interested people would
>>>>>>> split
>>>>>>> >
>>>>>>> > Statefun from the Flink project and make it a separate top level
>>>>>>> project
>>>>>>> > under the Apache Flink umbrella (similar as recently has happened
>>>>>>> with
>>>>>>> > Flink Table Store, which has become Apache Paimon).
>>>>>>> >
>>>>>>> > If we see no improvements in the coming period, we should consider
>>>>>>> >
>>>>>>> > sunsetting Statefun and communicate that clearly to the users.
>>>>>>> >
>>>>>>> > I'm looking forward to your thoughts.
>>>>>>> >
>>>>>>> > Best regards,
>>>>>>> >
>>>>>>> > Martijn
>>>>>>> >
>>>>>>> > [1] https://nightlies.apache.org/flink/flink-statefun-docs-master/
>>>>>>> <
>>>>>>> >
>>>>>>> > https://nightlies.apache.org/flink/flink-statefun-docs-master/>
>>>>>>> >
>>>>>>> > [2] https://github.com/apache/flink-statefun <
>>>>>>> >
>>>>>>> > https://github.com/apache/flink-statefun>
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>>
>>>>>>

Re: [DISCUSS] Status of Statefun Project

[Galen Warren <ga...@cvillewarrens.com.INVALID>]

Great -- thanks!

I'm going to be out of town for about a week but I'll take a look at this
when I'm back.

On Tue, Jun 20, 2023 at 8:46 AM Martijn Visser <mv...@confluent.io> wrote:

> Hi Galen,
>
> Yes, I'll be more than happy to help with Statefun releases.
>
> Best regards,
>
> Martijn
>
> On Tue, Jun 20, 2023 at 2:21 PM Galen Warren <ga...@cvillewarrens.com>
> wrote:
>
>> Thanks.
>>
>> Martijn, to answer your question, I'd need to do a small amount of work
>> to get a PR ready, but not much. Happy to do it if we're deciding to
>> restart Statefun releases -- are we?
>>
>> -- Galen
>>
>> On Sat, Jun 17, 2023 at 9:47 AM Tzu-Li (Gordon) Tai <tz...@apache.org>
>> wrote:
>>
>>> > Perhaps he could weigh in on whether the combination of automated
>>> tests plus those smoke tests should be sufficient for testing with new
>>> Flink versions
>>>
>>> What we usually did at the bare minimum for new StateFun releases was
>>> the following:
>>>
>>>    1. Build tests (including the smoke tests in the e2e module, which
>>>    covers important tests like exactly-once verification)
>>>    2. Updating the flink-statefun-playground repo and manually running
>>>    all language examples there.
>>>
>>> If upgrading Flink versions was the only change in the release, I'd
>>> probably say that this is sufficient.
>>>
>>> Best,
>>> Gordon
>>>
>>> On Thu, Jun 15, 2023 at 5:25 AM Martijn Visser <ma...@apache.org>
>>> wrote:
>>>
>>>> Let me know if you have a PR for a Flink update :)
>>>>
>>>> On Thu, Jun 8, 2023 at 5:52 PM Galen Warren via user <
>>>> user@flink.apache.org> wrote:
>>>>
>>>>> Thanks Martijn.
>>>>>
>>>>> Personally, I'm already using a local fork of Statefun that is
>>>>> compatible with Flink 1.16.x, so I wouldn't have any need for a released
>>>>> version compatible with 1.15.x. I'd be happy to do the PRs to modify
>>>>> Statefun to work with new versions of Flink as they come along.
>>>>>
>>>>> As for testing, Statefun does have unit tests and Gordon also sent me
>>>>> instructions a while back for how to do some additional smoke tests which
>>>>> are pretty straightforward. Perhaps he could weigh in on whether the
>>>>> combination of automated tests plus those smoke tests should be sufficient
>>>>> for testing with new Flink versions (I believe the answer is yes).
>>>>>
>>>>> -- Galen
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Jun 8, 2023 at 8:01 AM Martijn Visser <
>>>>> martijnvisser@apache.org> wrote:
>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> Apologies for the late reply.
>>>>>>
>>>>>> I'm willing to help out with merging requests in Statefun to keep them
>>>>>> compatible with new Flink releases and create new releases. I do
>>>>>> think that
>>>>>> validation of the functionality of these releases depends a lot on
>>>>>> those
>>>>>> who do these compatibility updates, with PMC members helping out with
>>>>>> the
>>>>>> formal process.
>>>>>>
>>>>>> > Why can't the Apache Software Foundation allow community members to
>>>>>> bring
>>>>>> it up to date?
>>>>>>
>>>>>> There's nothing preventing anyone from reviewing any of the current
>>>>>> PRs or
>>>>>> opening new ones. However, none of them are approved [1], so there's
>>>>>> also
>>>>>> nothing to merge.
>>>>>>
>>>>>> > I believe that there are people and companies on this mailing list
>>>>>> interested in supporting Apache Flink Stateful Functions.
>>>>>>
>>>>>> If so, then now is the time to show.
>>>>>>
>>>>>> Would there be a preference to create a release with Galen's merged
>>>>>> compatibility update to Flink 1.15.2, or do we want to skip that and
>>>>>> go
>>>>>> straight to a newer version?
>>>>>>
>>>>>> Best regards,
>>>>>>
>>>>>> Martijn
>>>>>>
>>>>>> [1]
>>>>>>
>>>>>> https://github.com/apache/flink-statefun/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved
>>>>>>
>>>>>> On Tue, Jun 6, 2023 at 3:55 PM Marco Villalobos <
>>>>>> mvillalobos@kineteque.com>
>>>>>> wrote:
>>>>>>
>>>>>> > Why can't the Apache Software Foundation allow community members to
>>>>>> bring
>>>>>> > it up to date?
>>>>>> >
>>>>>> > What's the process for that?
>>>>>> >
>>>>>> > I believe that there are people and companies on this mailing list
>>>>>> > interested in supporting Apache Flink Stateful Functions.
>>>>>> >
>>>>>> > You already had two people on this thread express interest.
>>>>>> >
>>>>>> > At the very least, we could keep the library versions up to date.
>>>>>> >
>>>>>> > There are only a small list of new features that might be
>>>>>> worthwhile:
>>>>>> >
>>>>>> > 1. event time processing
>>>>>> > 2. state rest api
>>>>>> >
>>>>>> >
>>>>>> > On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <ch...@apache.org>
>>>>>> wrote:
>>>>>> >
>>>>>> > If you were to fork it *and want to redistribute it* then the short
>>>>>> > version is that
>>>>>> >
>>>>>> >    1. you have to adhere to the Apache licensing requirements
>>>>>> >    2. you have to make it clear that your fork does not belong to
>>>>>> the
>>>>>> >    Apache Flink project. (Trademarks and all that)
>>>>>> >
>>>>>> > Neither should be significant hurdles (there should also be plenty
>>>>>> of
>>>>>> > online resources regarding 1), and if you do this then you can
>>>>>> freely share
>>>>>> > your fork with others.
>>>>>> >
>>>>>> > I've also pinged Martijn to take a look at this thread.
>>>>>> > To my knowledge the project hasn't decided anything yet.
>>>>>> >
>>>>>> > On 27/05/2023 04:05, Galen Warren wrote:
>>>>>> >
>>>>>> > Ok, I get it. No interest.
>>>>>> >
>>>>>> > If this project is being abandoned, I guess I'll work with my own
>>>>>> fork. Is
>>>>>> > there anything I should consider here? Can I share it with other
>>>>>> people who
>>>>>> > use this project?
>>>>>> >
>>>>>> > On Tue, May 16, 2023 at 10:50 AM Galen Warren <
>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
>>>>>> > wrote:
>>>>>> >
>>>>>> >
>>>>>> > Hi Martijn, since you opened this discussion thread, I'm curious
>>>>>> what your
>>>>>> > thoughts are in light of the responses? Thanks.
>>>>>> >
>>>>>> > On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <
>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
>>>>>> > wrote:
>>>>>> >
>>>>>> >
>>>>>> > I use Apache Flink for stream processing, and StateFun as a hand-off
>>>>>> >
>>>>>> > point for the rest of the application.
>>>>>> > It serves well as a bridge between a Flink Streaming job and
>>>>>> > micro-services.
>>>>>> >
>>>>>> > This is essentially how I use it as well, and I would also be sad
>>>>>> to see
>>>>>> > it sunsetted. It works well; I don't know that there is a lot of new
>>>>>> > development required, but if there are no new Statefun releases,
>>>>>> then
>>>>>> > Statefun can only be used with older Flink versions.
>>>>>> >
>>>>>> > On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
>>>>>> mvillalobos@kineteque.com> wrote:
>>>>>> >
>>>>>> >
>>>>>> > I am currently using Stateful Functions in my application.
>>>>>> >
>>>>>> > I use Apache Flink for stream processing, and StateFun as a hand-off
>>>>>> > point for the rest of the application.
>>>>>> > It serves well as a bridge between a Flink Streaming job and
>>>>>> > micro-services.
>>>>>> >
>>>>>> > I would be disappointed if StateFun was sunsetted.  Its a good idea.
>>>>>> >
>>>>>> > If there is anything I can do to help, as a contributor perhaps,
>>>>>> please
>>>>>> > let me know.
>>>>>> >
>>>>>> >
>>>>>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <
>>>>>> martijnvisser@apache.org> <ma...@apache.org>
>>>>>> >
>>>>>> > wrote:
>>>>>> >
>>>>>> > Hi everyone,
>>>>>> >
>>>>>> > I want to open a discussion on the status of the Statefun Project
>>>>>> [1]
>>>>>> >
>>>>>> > in Apache Flink. As you might have noticed, there hasn't been much
>>>>>> > development over the past months in the Statefun repository [2].
>>>>>> There is
>>>>>> > currently a lack of active contributors and committers who are able
>>>>>> to help
>>>>>> > with the maintenance of the project.
>>>>>> >
>>>>>> > In order to improve the situation, we need to solve the lack of
>>>>>> >
>>>>>> > committers and the lack of contributors.
>>>>>> >
>>>>>> > On the lack of committers:
>>>>>> >
>>>>>> > 1. Ideally, there are some of the current Flink committers who have
>>>>>> >
>>>>>> > the bandwidth and can help with reviewing PRs and merging them.
>>>>>> >
>>>>>> > 2. If that's not an option, it could be a consideration that current
>>>>>> >
>>>>>> > committers only approve and review PRs, that are approved by those
>>>>>> who are
>>>>>> > willing to contribute to Statefun and if the CI passes
>>>>>> >
>>>>>> > On the lack of contributors:
>>>>>> >
>>>>>> > 3. Next to having this discussion on the Dev and User mailing list,
>>>>>> we
>>>>>> >
>>>>>> > can also create a blog with a call for new contributors on the Flink
>>>>>> > project website, send out some tweets on the Flink / Statefun
>>>>>> twitter
>>>>>> > accounts, post messages on Slack etc. In that message, we would
>>>>>> inform how
>>>>>> > those that are interested in contributing can start and where they
>>>>>> could
>>>>>> > reach out for more information.
>>>>>> >
>>>>>> > There's also option 4. where a group of interested people would
>>>>>> split
>>>>>> >
>>>>>> > Statefun from the Flink project and make it a separate top level
>>>>>> project
>>>>>> > under the Apache Flink umbrella (similar as recently has happened
>>>>>> with
>>>>>> > Flink Table Store, which has become Apache Paimon).
>>>>>> >
>>>>>> > If we see no improvements in the coming period, we should consider
>>>>>> >
>>>>>> > sunsetting Statefun and communicate that clearly to the users.
>>>>>> >
>>>>>> > I'm looking forward to your thoughts.
>>>>>> >
>>>>>> > Best regards,
>>>>>> >
>>>>>> > Martijn
>>>>>> >
>>>>>> > [1] https://nightlies.apache.org/flink/flink-statefun-docs-master/
>>>>>> <
>>>>>> >
>>>>>> > https://nightlies.apache.org/flink/flink-statefun-docs-master/>
>>>>>> >
>>>>>> > [2] https://github.com/apache/flink-statefun <
>>>>>> >
>>>>>> > https://github.com/apache/flink-statefun>
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>>
>>>>>

Re: [DISCUSS] Status of Statefun Project

[Galen Warren via user <us...@flink.apache.org>]

Great -- thanks!

I'm going to be out of town for about a week but I'll take a look at this
when I'm back.

On Tue, Jun 20, 2023 at 8:46 AM Martijn Visser <mv...@confluent.io> wrote:

> Hi Galen,
>
> Yes, I'll be more than happy to help with Statefun releases.
>
> Best regards,
>
> Martijn
>
> On Tue, Jun 20, 2023 at 2:21 PM Galen Warren <ga...@cvillewarrens.com>
> wrote:
>
>> Thanks.
>>
>> Martijn, to answer your question, I'd need to do a small amount of work
>> to get a PR ready, but not much. Happy to do it if we're deciding to
>> restart Statefun releases -- are we?
>>
>> -- Galen
>>
>> On Sat, Jun 17, 2023 at 9:47 AM Tzu-Li (Gordon) Tai <tz...@apache.org>
>> wrote:
>>
>>> > Perhaps he could weigh in on whether the combination of automated
>>> tests plus those smoke tests should be sufficient for testing with new
>>> Flink versions
>>>
>>> What we usually did at the bare minimum for new StateFun releases was
>>> the following:
>>>
>>>    1. Build tests (including the smoke tests in the e2e module, which
>>>    covers important tests like exactly-once verification)
>>>    2. Updating the flink-statefun-playground repo and manually running
>>>    all language examples there.
>>>
>>> If upgrading Flink versions was the only change in the release, I'd
>>> probably say that this is sufficient.
>>>
>>> Best,
>>> Gordon
>>>
>>> On Thu, Jun 15, 2023 at 5:25 AM Martijn Visser <ma...@apache.org>
>>> wrote:
>>>
>>>> Let me know if you have a PR for a Flink update :)
>>>>
>>>> On Thu, Jun 8, 2023 at 5:52 PM Galen Warren via user <
>>>> user@flink.apache.org> wrote:
>>>>
>>>>> Thanks Martijn.
>>>>>
>>>>> Personally, I'm already using a local fork of Statefun that is
>>>>> compatible with Flink 1.16.x, so I wouldn't have any need for a released
>>>>> version compatible with 1.15.x. I'd be happy to do the PRs to modify
>>>>> Statefun to work with new versions of Flink as they come along.
>>>>>
>>>>> As for testing, Statefun does have unit tests and Gordon also sent me
>>>>> instructions a while back for how to do some additional smoke tests which
>>>>> are pretty straightforward. Perhaps he could weigh in on whether the
>>>>> combination of automated tests plus those smoke tests should be sufficient
>>>>> for testing with new Flink versions (I believe the answer is yes).
>>>>>
>>>>> -- Galen
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Jun 8, 2023 at 8:01 AM Martijn Visser <
>>>>> martijnvisser@apache.org> wrote:
>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> Apologies for the late reply.
>>>>>>
>>>>>> I'm willing to help out with merging requests in Statefun to keep them
>>>>>> compatible with new Flink releases and create new releases. I do
>>>>>> think that
>>>>>> validation of the functionality of these releases depends a lot on
>>>>>> those
>>>>>> who do these compatibility updates, with PMC members helping out with
>>>>>> the
>>>>>> formal process.
>>>>>>
>>>>>> > Why can't the Apache Software Foundation allow community members to
>>>>>> bring
>>>>>> it up to date?
>>>>>>
>>>>>> There's nothing preventing anyone from reviewing any of the current
>>>>>> PRs or
>>>>>> opening new ones. However, none of them are approved [1], so there's
>>>>>> also
>>>>>> nothing to merge.
>>>>>>
>>>>>> > I believe that there are people and companies on this mailing list
>>>>>> interested in supporting Apache Flink Stateful Functions.
>>>>>>
>>>>>> If so, then now is the time to show.
>>>>>>
>>>>>> Would there be a preference to create a release with Galen's merged
>>>>>> compatibility update to Flink 1.15.2, or do we want to skip that and
>>>>>> go
>>>>>> straight to a newer version?
>>>>>>
>>>>>> Best regards,
>>>>>>
>>>>>> Martijn
>>>>>>
>>>>>> [1]
>>>>>>
>>>>>> https://github.com/apache/flink-statefun/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved
>>>>>>
>>>>>> On Tue, Jun 6, 2023 at 3:55 PM Marco Villalobos <
>>>>>> mvillalobos@kineteque.com>
>>>>>> wrote:
>>>>>>
>>>>>> > Why can't the Apache Software Foundation allow community members to
>>>>>> bring
>>>>>> > it up to date?
>>>>>> >
>>>>>> > What's the process for that?
>>>>>> >
>>>>>> > I believe that there are people and companies on this mailing list
>>>>>> > interested in supporting Apache Flink Stateful Functions.
>>>>>> >
>>>>>> > You already had two people on this thread express interest.
>>>>>> >
>>>>>> > At the very least, we could keep the library versions up to date.
>>>>>> >
>>>>>> > There are only a small list of new features that might be
>>>>>> worthwhile:
>>>>>> >
>>>>>> > 1. event time processing
>>>>>> > 2. state rest api
>>>>>> >
>>>>>> >
>>>>>> > On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <ch...@apache.org>
>>>>>> wrote:
>>>>>> >
>>>>>> > If you were to fork it *and want to redistribute it* then the short
>>>>>> > version is that
>>>>>> >
>>>>>> >    1. you have to adhere to the Apache licensing requirements
>>>>>> >    2. you have to make it clear that your fork does not belong to
>>>>>> the
>>>>>> >    Apache Flink project. (Trademarks and all that)
>>>>>> >
>>>>>> > Neither should be significant hurdles (there should also be plenty
>>>>>> of
>>>>>> > online resources regarding 1), and if you do this then you can
>>>>>> freely share
>>>>>> > your fork with others.
>>>>>> >
>>>>>> > I've also pinged Martijn to take a look at this thread.
>>>>>> > To my knowledge the project hasn't decided anything yet.
>>>>>> >
>>>>>> > On 27/05/2023 04:05, Galen Warren wrote:
>>>>>> >
>>>>>> > Ok, I get it. No interest.
>>>>>> >
>>>>>> > If this project is being abandoned, I guess I'll work with my own
>>>>>> fork. Is
>>>>>> > there anything I should consider here? Can I share it with other
>>>>>> people who
>>>>>> > use this project?
>>>>>> >
>>>>>> > On Tue, May 16, 2023 at 10:50 AM Galen Warren <
>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
>>>>>> > wrote:
>>>>>> >
>>>>>> >
>>>>>> > Hi Martijn, since you opened this discussion thread, I'm curious
>>>>>> what your
>>>>>> > thoughts are in light of the responses? Thanks.
>>>>>> >
>>>>>> > On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <
>>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
>>>>>> > wrote:
>>>>>> >
>>>>>> >
>>>>>> > I use Apache Flink for stream processing, and StateFun as a hand-off
>>>>>> >
>>>>>> > point for the rest of the application.
>>>>>> > It serves well as a bridge between a Flink Streaming job and
>>>>>> > micro-services.
>>>>>> >
>>>>>> > This is essentially how I use it as well, and I would also be sad
>>>>>> to see
>>>>>> > it sunsetted. It works well; I don't know that there is a lot of new
>>>>>> > development required, but if there are no new Statefun releases,
>>>>>> then
>>>>>> > Statefun can only be used with older Flink versions.
>>>>>> >
>>>>>> > On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
>>>>>> mvillalobos@kineteque.com> wrote:
>>>>>> >
>>>>>> >
>>>>>> > I am currently using Stateful Functions in my application.
>>>>>> >
>>>>>> > I use Apache Flink for stream processing, and StateFun as a hand-off
>>>>>> > point for the rest of the application.
>>>>>> > It serves well as a bridge between a Flink Streaming job and
>>>>>> > micro-services.
>>>>>> >
>>>>>> > I would be disappointed if StateFun was sunsetted.  Its a good idea.
>>>>>> >
>>>>>> > If there is anything I can do to help, as a contributor perhaps,
>>>>>> please
>>>>>> > let me know.
>>>>>> >
>>>>>> >
>>>>>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <
>>>>>> martijnvisser@apache.org> <ma...@apache.org>
>>>>>> >
>>>>>> > wrote:
>>>>>> >
>>>>>> > Hi everyone,
>>>>>> >
>>>>>> > I want to open a discussion on the status of the Statefun Project
>>>>>> [1]
>>>>>> >
>>>>>> > in Apache Flink. As you might have noticed, there hasn't been much
>>>>>> > development over the past months in the Statefun repository [2].
>>>>>> There is
>>>>>> > currently a lack of active contributors and committers who are able
>>>>>> to help
>>>>>> > with the maintenance of the project.
>>>>>> >
>>>>>> > In order to improve the situation, we need to solve the lack of
>>>>>> >
>>>>>> > committers and the lack of contributors.
>>>>>> >
>>>>>> > On the lack of committers:
>>>>>> >
>>>>>> > 1. Ideally, there are some of the current Flink committers who have
>>>>>> >
>>>>>> > the bandwidth and can help with reviewing PRs and merging them.
>>>>>> >
>>>>>> > 2. If that's not an option, it could be a consideration that current
>>>>>> >
>>>>>> > committers only approve and review PRs, that are approved by those
>>>>>> who are
>>>>>> > willing to contribute to Statefun and if the CI passes
>>>>>> >
>>>>>> > On the lack of contributors:
>>>>>> >
>>>>>> > 3. Next to having this discussion on the Dev and User mailing list,
>>>>>> we
>>>>>> >
>>>>>> > can also create a blog with a call for new contributors on the Flink
>>>>>> > project website, send out some tweets on the Flink / Statefun
>>>>>> twitter
>>>>>> > accounts, post messages on Slack etc. In that message, we would
>>>>>> inform how
>>>>>> > those that are interested in contributing can start and where they
>>>>>> could
>>>>>> > reach out for more information.
>>>>>> >
>>>>>> > There's also option 4. where a group of interested people would
>>>>>> split
>>>>>> >
>>>>>> > Statefun from the Flink project and make it a separate top level
>>>>>> project
>>>>>> > under the Apache Flink umbrella (similar as recently has happened
>>>>>> with
>>>>>> > Flink Table Store, which has become Apache Paimon).
>>>>>> >
>>>>>> > If we see no improvements in the coming period, we should consider
>>>>>> >
>>>>>> > sunsetting Statefun and communicate that clearly to the users.
>>>>>> >
>>>>>> > I'm looking forward to your thoughts.
>>>>>> >
>>>>>> > Best regards,
>>>>>> >
>>>>>> > Martijn
>>>>>> >
>>>>>> > [1] https://nightlies.apache.org/flink/flink-statefun-docs-master/
>>>>>> <
>>>>>> >
>>>>>> > https://nightlies.apache.org/flink/flink-statefun-docs-master/>
>>>>>> >
>>>>>> > [2] https://github.com/apache/flink-statefun <
>>>>>> >
>>>>>> > https://github.com/apache/flink-statefun>
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>>
>>>>>

Re: [DISCUSS] Status of Statefun Project

[Martijn Visser via user <us...@flink.apache.org>]

Hi Galen,

Yes, I'll be more than happy to help with Statefun releases.

Best regards,

Martijn

On Tue, Jun 20, 2023 at 2:21 PM Galen Warren <ga...@cvillewarrens.com>
wrote:

> Thanks.
>
> Martijn, to answer your question, I'd need to do a small amount of work to
> get a PR ready, but not much. Happy to do it if we're deciding to restart
> Statefun releases -- are we?
>
> -- Galen
>
> On Sat, Jun 17, 2023 at 9:47 AM Tzu-Li (Gordon) Tai <tz...@apache.org>
> wrote:
>
>> > Perhaps he could weigh in on whether the combination of automated tests
>> plus those smoke tests should be sufficient for testing with new Flink
>> versions
>>
>> What we usually did at the bare minimum for new StateFun releases was the
>> following:
>>
>>    1. Build tests (including the smoke tests in the e2e module, which
>>    covers important tests like exactly-once verification)
>>    2. Updating the flink-statefun-playground repo and manually running
>>    all language examples there.
>>
>> If upgrading Flink versions was the only change in the release, I'd
>> probably say that this is sufficient.
>>
>> Best,
>> Gordon
>>
>> On Thu, Jun 15, 2023 at 5:25 AM Martijn Visser <ma...@apache.org>
>> wrote:
>>
>>> Let me know if you have a PR for a Flink update :)
>>>
>>> On Thu, Jun 8, 2023 at 5:52 PM Galen Warren via user <
>>> user@flink.apache.org> wrote:
>>>
>>>> Thanks Martijn.
>>>>
>>>> Personally, I'm already using a local fork of Statefun that is
>>>> compatible with Flink 1.16.x, so I wouldn't have any need for a released
>>>> version compatible with 1.15.x. I'd be happy to do the PRs to modify
>>>> Statefun to work with new versions of Flink as they come along.
>>>>
>>>> As for testing, Statefun does have unit tests and Gordon also sent me
>>>> instructions a while back for how to do some additional smoke tests which
>>>> are pretty straightforward. Perhaps he could weigh in on whether the
>>>> combination of automated tests plus those smoke tests should be sufficient
>>>> for testing with new Flink versions (I believe the answer is yes).
>>>>
>>>> -- Galen
>>>>
>>>>
>>>>
>>>> On Thu, Jun 8, 2023 at 8:01 AM Martijn Visser <ma...@apache.org>
>>>> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> Apologies for the late reply.
>>>>>
>>>>> I'm willing to help out with merging requests in Statefun to keep them
>>>>> compatible with new Flink releases and create new releases. I do think
>>>>> that
>>>>> validation of the functionality of these releases depends a lot on
>>>>> those
>>>>> who do these compatibility updates, with PMC members helping out with
>>>>> the
>>>>> formal process.
>>>>>
>>>>> > Why can't the Apache Software Foundation allow community members to
>>>>> bring
>>>>> it up to date?
>>>>>
>>>>> There's nothing preventing anyone from reviewing any of the current
>>>>> PRs or
>>>>> opening new ones. However, none of them are approved [1], so there's
>>>>> also
>>>>> nothing to merge.
>>>>>
>>>>> > I believe that there are people and companies on this mailing list
>>>>> interested in supporting Apache Flink Stateful Functions.
>>>>>
>>>>> If so, then now is the time to show.
>>>>>
>>>>> Would there be a preference to create a release with Galen's merged
>>>>> compatibility update to Flink 1.15.2, or do we want to skip that and go
>>>>> straight to a newer version?
>>>>>
>>>>> Best regards,
>>>>>
>>>>> Martijn
>>>>>
>>>>> [1]
>>>>>
>>>>> https://github.com/apache/flink-statefun/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved
>>>>>
>>>>> On Tue, Jun 6, 2023 at 3:55 PM Marco Villalobos <
>>>>> mvillalobos@kineteque.com>
>>>>> wrote:
>>>>>
>>>>> > Why can't the Apache Software Foundation allow community members to
>>>>> bring
>>>>> > it up to date?
>>>>> >
>>>>> > What's the process for that?
>>>>> >
>>>>> > I believe that there are people and companies on this mailing list
>>>>> > interested in supporting Apache Flink Stateful Functions.
>>>>> >
>>>>> > You already had two people on this thread express interest.
>>>>> >
>>>>> > At the very least, we could keep the library versions up to date.
>>>>> >
>>>>> > There are only a small list of new features that might be worthwhile:
>>>>> >
>>>>> > 1. event time processing
>>>>> > 2. state rest api
>>>>> >
>>>>> >
>>>>> > On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <ch...@apache.org>
>>>>> wrote:
>>>>> >
>>>>> > If you were to fork it *and want to redistribute it* then the short
>>>>> > version is that
>>>>> >
>>>>> >    1. you have to adhere to the Apache licensing requirements
>>>>> >    2. you have to make it clear that your fork does not belong to the
>>>>> >    Apache Flink project. (Trademarks and all that)
>>>>> >
>>>>> > Neither should be significant hurdles (there should also be plenty of
>>>>> > online resources regarding 1), and if you do this then you can
>>>>> freely share
>>>>> > your fork with others.
>>>>> >
>>>>> > I've also pinged Martijn to take a look at this thread.
>>>>> > To my knowledge the project hasn't decided anything yet.
>>>>> >
>>>>> > On 27/05/2023 04:05, Galen Warren wrote:
>>>>> >
>>>>> > Ok, I get it. No interest.
>>>>> >
>>>>> > If this project is being abandoned, I guess I'll work with my own
>>>>> fork. Is
>>>>> > there anything I should consider here? Can I share it with other
>>>>> people who
>>>>> > use this project?
>>>>> >
>>>>> > On Tue, May 16, 2023 at 10:50 AM Galen Warren <
>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
>>>>> > wrote:
>>>>> >
>>>>> >
>>>>> > Hi Martijn, since you opened this discussion thread, I'm curious
>>>>> what your
>>>>> > thoughts are in light of the responses? Thanks.
>>>>> >
>>>>> > On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <
>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
>>>>> > wrote:
>>>>> >
>>>>> >
>>>>> > I use Apache Flink for stream processing, and StateFun as a hand-off
>>>>> >
>>>>> > point for the rest of the application.
>>>>> > It serves well as a bridge between a Flink Streaming job and
>>>>> > micro-services.
>>>>> >
>>>>> > This is essentially how I use it as well, and I would also be sad to
>>>>> see
>>>>> > it sunsetted. It works well; I don't know that there is a lot of new
>>>>> > development required, but if there are no new Statefun releases, then
>>>>> > Statefun can only be used with older Flink versions.
>>>>> >
>>>>> > On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
>>>>> mvillalobos@kineteque.com> wrote:
>>>>> >
>>>>> >
>>>>> > I am currently using Stateful Functions in my application.
>>>>> >
>>>>> > I use Apache Flink for stream processing, and StateFun as a hand-off
>>>>> > point for the rest of the application.
>>>>> > It serves well as a bridge between a Flink Streaming job and
>>>>> > micro-services.
>>>>> >
>>>>> > I would be disappointed if StateFun was sunsetted.  Its a good idea.
>>>>> >
>>>>> > If there is anything I can do to help, as a contributor perhaps,
>>>>> please
>>>>> > let me know.
>>>>> >
>>>>> >
>>>>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <ma...@apache.org>
>>>>> <ma...@apache.org>
>>>>> >
>>>>> > wrote:
>>>>> >
>>>>> > Hi everyone,
>>>>> >
>>>>> > I want to open a discussion on the status of the Statefun Project [1]
>>>>> >
>>>>> > in Apache Flink. As you might have noticed, there hasn't been much
>>>>> > development over the past months in the Statefun repository [2].
>>>>> There is
>>>>> > currently a lack of active contributors and committers who are able
>>>>> to help
>>>>> > with the maintenance of the project.
>>>>> >
>>>>> > In order to improve the situation, we need to solve the lack of
>>>>> >
>>>>> > committers and the lack of contributors.
>>>>> >
>>>>> > On the lack of committers:
>>>>> >
>>>>> > 1. Ideally, there are some of the current Flink committers who have
>>>>> >
>>>>> > the bandwidth and can help with reviewing PRs and merging them.
>>>>> >
>>>>> > 2. If that's not an option, it could be a consideration that current
>>>>> >
>>>>> > committers only approve and review PRs, that are approved by those
>>>>> who are
>>>>> > willing to contribute to Statefun and if the CI passes
>>>>> >
>>>>> > On the lack of contributors:
>>>>> >
>>>>> > 3. Next to having this discussion on the Dev and User mailing list,
>>>>> we
>>>>> >
>>>>> > can also create a blog with a call for new contributors on the Flink
>>>>> > project website, send out some tweets on the Flink / Statefun twitter
>>>>> > accounts, post messages on Slack etc. In that message, we would
>>>>> inform how
>>>>> > those that are interested in contributing can start and where they
>>>>> could
>>>>> > reach out for more information.
>>>>> >
>>>>> > There's also option 4. where a group of interested people would split
>>>>> >
>>>>> > Statefun from the Flink project and make it a separate top level
>>>>> project
>>>>> > under the Apache Flink umbrella (similar as recently has happened
>>>>> with
>>>>> > Flink Table Store, which has become Apache Paimon).
>>>>> >
>>>>> > If we see no improvements in the coming period, we should consider
>>>>> >
>>>>> > sunsetting Statefun and communicate that clearly to the users.
>>>>> >
>>>>> > I'm looking forward to your thoughts.
>>>>> >
>>>>> > Best regards,
>>>>> >
>>>>> > Martijn
>>>>> >
>>>>> > [1] https://nightlies.apache.org/flink/flink-statefun-docs-master/ <
>>>>> >
>>>>> > https://nightlies.apache.org/flink/flink-statefun-docs-master/>
>>>>> >
>>>>> > [2] https://github.com/apache/flink-statefun <
>>>>> >
>>>>> > https://github.com/apache/flink-statefun>
>>>>> >
>>>>> >
>>>>> >
>>>>> >
>>>>> >
>>>>>
>>>>

Re: [DISCUSS] Status of Statefun Project

[Martijn Visser <mv...@confluent.io.INVALID>]

Hi Galen,

Yes, I'll be more than happy to help with Statefun releases.

Best regards,

Martijn

On Tue, Jun 20, 2023 at 2:21 PM Galen Warren <ga...@cvillewarrens.com>
wrote:

> Thanks.
>
> Martijn, to answer your question, I'd need to do a small amount of work to
> get a PR ready, but not much. Happy to do it if we're deciding to restart
> Statefun releases -- are we?
>
> -- Galen
>
> On Sat, Jun 17, 2023 at 9:47 AM Tzu-Li (Gordon) Tai <tz...@apache.org>
> wrote:
>
>> > Perhaps he could weigh in on whether the combination of automated tests
>> plus those smoke tests should be sufficient for testing with new Flink
>> versions
>>
>> What we usually did at the bare minimum for new StateFun releases was the
>> following:
>>
>>    1. Build tests (including the smoke tests in the e2e module, which
>>    covers important tests like exactly-once verification)
>>    2. Updating the flink-statefun-playground repo and manually running
>>    all language examples there.
>>
>> If upgrading Flink versions was the only change in the release, I'd
>> probably say that this is sufficient.
>>
>> Best,
>> Gordon
>>
>> On Thu, Jun 15, 2023 at 5:25 AM Martijn Visser <ma...@apache.org>
>> wrote:
>>
>>> Let me know if you have a PR for a Flink update :)
>>>
>>> On Thu, Jun 8, 2023 at 5:52 PM Galen Warren via user <
>>> user@flink.apache.org> wrote:
>>>
>>>> Thanks Martijn.
>>>>
>>>> Personally, I'm already using a local fork of Statefun that is
>>>> compatible with Flink 1.16.x, so I wouldn't have any need for a released
>>>> version compatible with 1.15.x. I'd be happy to do the PRs to modify
>>>> Statefun to work with new versions of Flink as they come along.
>>>>
>>>> As for testing, Statefun does have unit tests and Gordon also sent me
>>>> instructions a while back for how to do some additional smoke tests which
>>>> are pretty straightforward. Perhaps he could weigh in on whether the
>>>> combination of automated tests plus those smoke tests should be sufficient
>>>> for testing with new Flink versions (I believe the answer is yes).
>>>>
>>>> -- Galen
>>>>
>>>>
>>>>
>>>> On Thu, Jun 8, 2023 at 8:01 AM Martijn Visser <ma...@apache.org>
>>>> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> Apologies for the late reply.
>>>>>
>>>>> I'm willing to help out with merging requests in Statefun to keep them
>>>>> compatible with new Flink releases and create new releases. I do think
>>>>> that
>>>>> validation of the functionality of these releases depends a lot on
>>>>> those
>>>>> who do these compatibility updates, with PMC members helping out with
>>>>> the
>>>>> formal process.
>>>>>
>>>>> > Why can't the Apache Software Foundation allow community members to
>>>>> bring
>>>>> it up to date?
>>>>>
>>>>> There's nothing preventing anyone from reviewing any of the current
>>>>> PRs or
>>>>> opening new ones. However, none of them are approved [1], so there's
>>>>> also
>>>>> nothing to merge.
>>>>>
>>>>> > I believe that there are people and companies on this mailing list
>>>>> interested in supporting Apache Flink Stateful Functions.
>>>>>
>>>>> If so, then now is the time to show.
>>>>>
>>>>> Would there be a preference to create a release with Galen's merged
>>>>> compatibility update to Flink 1.15.2, or do we want to skip that and go
>>>>> straight to a newer version?
>>>>>
>>>>> Best regards,
>>>>>
>>>>> Martijn
>>>>>
>>>>> [1]
>>>>>
>>>>> https://github.com/apache/flink-statefun/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved
>>>>>
>>>>> On Tue, Jun 6, 2023 at 3:55 PM Marco Villalobos <
>>>>> mvillalobos@kineteque.com>
>>>>> wrote:
>>>>>
>>>>> > Why can't the Apache Software Foundation allow community members to
>>>>> bring
>>>>> > it up to date?
>>>>> >
>>>>> > What's the process for that?
>>>>> >
>>>>> > I believe that there are people and companies on this mailing list
>>>>> > interested in supporting Apache Flink Stateful Functions.
>>>>> >
>>>>> > You already had two people on this thread express interest.
>>>>> >
>>>>> > At the very least, we could keep the library versions up to date.
>>>>> >
>>>>> > There are only a small list of new features that might be worthwhile:
>>>>> >
>>>>> > 1. event time processing
>>>>> > 2. state rest api
>>>>> >
>>>>> >
>>>>> > On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <ch...@apache.org>
>>>>> wrote:
>>>>> >
>>>>> > If you were to fork it *and want to redistribute it* then the short
>>>>> > version is that
>>>>> >
>>>>> >    1. you have to adhere to the Apache licensing requirements
>>>>> >    2. you have to make it clear that your fork does not belong to the
>>>>> >    Apache Flink project. (Trademarks and all that)
>>>>> >
>>>>> > Neither should be significant hurdles (there should also be plenty of
>>>>> > online resources regarding 1), and if you do this then you can
>>>>> freely share
>>>>> > your fork with others.
>>>>> >
>>>>> > I've also pinged Martijn to take a look at this thread.
>>>>> > To my knowledge the project hasn't decided anything yet.
>>>>> >
>>>>> > On 27/05/2023 04:05, Galen Warren wrote:
>>>>> >
>>>>> > Ok, I get it. No interest.
>>>>> >
>>>>> > If this project is being abandoned, I guess I'll work with my own
>>>>> fork. Is
>>>>> > there anything I should consider here? Can I share it with other
>>>>> people who
>>>>> > use this project?
>>>>> >
>>>>> > On Tue, May 16, 2023 at 10:50 AM Galen Warren <
>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
>>>>> > wrote:
>>>>> >
>>>>> >
>>>>> > Hi Martijn, since you opened this discussion thread, I'm curious
>>>>> what your
>>>>> > thoughts are in light of the responses? Thanks.
>>>>> >
>>>>> > On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <
>>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
>>>>> > wrote:
>>>>> >
>>>>> >
>>>>> > I use Apache Flink for stream processing, and StateFun as a hand-off
>>>>> >
>>>>> > point for the rest of the application.
>>>>> > It serves well as a bridge between a Flink Streaming job and
>>>>> > micro-services.
>>>>> >
>>>>> > This is essentially how I use it as well, and I would also be sad to
>>>>> see
>>>>> > it sunsetted. It works well; I don't know that there is a lot of new
>>>>> > development required, but if there are no new Statefun releases, then
>>>>> > Statefun can only be used with older Flink versions.
>>>>> >
>>>>> > On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
>>>>> mvillalobos@kineteque.com> wrote:
>>>>> >
>>>>> >
>>>>> > I am currently using Stateful Functions in my application.
>>>>> >
>>>>> > I use Apache Flink for stream processing, and StateFun as a hand-off
>>>>> > point for the rest of the application.
>>>>> > It serves well as a bridge between a Flink Streaming job and
>>>>> > micro-services.
>>>>> >
>>>>> > I would be disappointed if StateFun was sunsetted.  Its a good idea.
>>>>> >
>>>>> > If there is anything I can do to help, as a contributor perhaps,
>>>>> please
>>>>> > let me know.
>>>>> >
>>>>> >
>>>>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <ma...@apache.org>
>>>>> <ma...@apache.org>
>>>>> >
>>>>> > wrote:
>>>>> >
>>>>> > Hi everyone,
>>>>> >
>>>>> > I want to open a discussion on the status of the Statefun Project [1]
>>>>> >
>>>>> > in Apache Flink. As you might have noticed, there hasn't been much
>>>>> > development over the past months in the Statefun repository [2].
>>>>> There is
>>>>> > currently a lack of active contributors and committers who are able
>>>>> to help
>>>>> > with the maintenance of the project.
>>>>> >
>>>>> > In order to improve the situation, we need to solve the lack of
>>>>> >
>>>>> > committers and the lack of contributors.
>>>>> >
>>>>> > On the lack of committers:
>>>>> >
>>>>> > 1. Ideally, there are some of the current Flink committers who have
>>>>> >
>>>>> > the bandwidth and can help with reviewing PRs and merging them.
>>>>> >
>>>>> > 2. If that's not an option, it could be a consideration that current
>>>>> >
>>>>> > committers only approve and review PRs, that are approved by those
>>>>> who are
>>>>> > willing to contribute to Statefun and if the CI passes
>>>>> >
>>>>> > On the lack of contributors:
>>>>> >
>>>>> > 3. Next to having this discussion on the Dev and User mailing list,
>>>>> we
>>>>> >
>>>>> > can also create a blog with a call for new contributors on the Flink
>>>>> > project website, send out some tweets on the Flink / Statefun twitter
>>>>> > accounts, post messages on Slack etc. In that message, we would
>>>>> inform how
>>>>> > those that are interested in contributing can start and where they
>>>>> could
>>>>> > reach out for more information.
>>>>> >
>>>>> > There's also option 4. where a group of interested people would split
>>>>> >
>>>>> > Statefun from the Flink project and make it a separate top level
>>>>> project
>>>>> > under the Apache Flink umbrella (similar as recently has happened
>>>>> with
>>>>> > Flink Table Store, which has become Apache Paimon).
>>>>> >
>>>>> > If we see no improvements in the coming period, we should consider
>>>>> >
>>>>> > sunsetting Statefun and communicate that clearly to the users.
>>>>> >
>>>>> > I'm looking forward to your thoughts.
>>>>> >
>>>>> > Best regards,
>>>>> >
>>>>> > Martijn
>>>>> >
>>>>> > [1] https://nightlies.apache.org/flink/flink-statefun-docs-master/ <
>>>>> >
>>>>> > https://nightlies.apache.org/flink/flink-statefun-docs-master/>
>>>>> >
>>>>> > [2] https://github.com/apache/flink-statefun <
>>>>> >
>>>>> > https://github.com/apache/flink-statefun>
>>>>> >
>>>>> >
>>>>> >
>>>>> >
>>>>> >
>>>>>
>>>>

Re: [DISCUSS] Status of Statefun Project

[Galen Warren <ga...@cvillewarrens.com.INVALID>]

Thanks.

Martijn, to answer your question, I'd need to do a small amount of work to
get a PR ready, but not much. Happy to do it if we're deciding to restart
Statefun releases -- are we?

-- Galen

On Sat, Jun 17, 2023 at 9:47 AM Tzu-Li (Gordon) Tai <tz...@apache.org>
wrote:

> > Perhaps he could weigh in on whether the combination of automated tests
> plus those smoke tests should be sufficient for testing with new Flink
> versions
>
> What we usually did at the bare minimum for new StateFun releases was the
> following:
>
>    1. Build tests (including the smoke tests in the e2e module, which
>    covers important tests like exactly-once verification)
>    2. Updating the flink-statefun-playground repo and manually running
>    all language examples there.
>
> If upgrading Flink versions was the only change in the release, I'd
> probably say that this is sufficient.
>
> Best,
> Gordon
>
> On Thu, Jun 15, 2023 at 5:25 AM Martijn Visser <ma...@apache.org>
> wrote:
>
>> Let me know if you have a PR for a Flink update :)
>>
>> On Thu, Jun 8, 2023 at 5:52 PM Galen Warren via user <
>> user@flink.apache.org> wrote:
>>
>>> Thanks Martijn.
>>>
>>> Personally, I'm already using a local fork of Statefun that is
>>> compatible with Flink 1.16.x, so I wouldn't have any need for a released
>>> version compatible with 1.15.x. I'd be happy to do the PRs to modify
>>> Statefun to work with new versions of Flink as they come along.
>>>
>>> As for testing, Statefun does have unit tests and Gordon also sent me
>>> instructions a while back for how to do some additional smoke tests which
>>> are pretty straightforward. Perhaps he could weigh in on whether the
>>> combination of automated tests plus those smoke tests should be sufficient
>>> for testing with new Flink versions (I believe the answer is yes).
>>>
>>> -- Galen
>>>
>>>
>>>
>>> On Thu, Jun 8, 2023 at 8:01 AM Martijn Visser <ma...@apache.org>
>>> wrote:
>>>
>>>> Hi all,
>>>>
>>>> Apologies for the late reply.
>>>>
>>>> I'm willing to help out with merging requests in Statefun to keep them
>>>> compatible with new Flink releases and create new releases. I do think
>>>> that
>>>> validation of the functionality of these releases depends a lot on those
>>>> who do these compatibility updates, with PMC members helping out with
>>>> the
>>>> formal process.
>>>>
>>>> > Why can't the Apache Software Foundation allow community members to
>>>> bring
>>>> it up to date?
>>>>
>>>> There's nothing preventing anyone from reviewing any of the current PRs
>>>> or
>>>> opening new ones. However, none of them are approved [1], so there's
>>>> also
>>>> nothing to merge.
>>>>
>>>> > I believe that there are people and companies on this mailing list
>>>> interested in supporting Apache Flink Stateful Functions.
>>>>
>>>> If so, then now is the time to show.
>>>>
>>>> Would there be a preference to create a release with Galen's merged
>>>> compatibility update to Flink 1.15.2, or do we want to skip that and go
>>>> straight to a newer version?
>>>>
>>>> Best regards,
>>>>
>>>> Martijn
>>>>
>>>> [1]
>>>>
>>>> https://github.com/apache/flink-statefun/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved
>>>>
>>>> On Tue, Jun 6, 2023 at 3:55 PM Marco Villalobos <
>>>> mvillalobos@kineteque.com>
>>>> wrote:
>>>>
>>>> > Why can't the Apache Software Foundation allow community members to
>>>> bring
>>>> > it up to date?
>>>> >
>>>> > What's the process for that?
>>>> >
>>>> > I believe that there are people and companies on this mailing list
>>>> > interested in supporting Apache Flink Stateful Functions.
>>>> >
>>>> > You already had two people on this thread express interest.
>>>> >
>>>> > At the very least, we could keep the library versions up to date.
>>>> >
>>>> > There are only a small list of new features that might be worthwhile:
>>>> >
>>>> > 1. event time processing
>>>> > 2. state rest api
>>>> >
>>>> >
>>>> > On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <ch...@apache.org>
>>>> wrote:
>>>> >
>>>> > If you were to fork it *and want to redistribute it* then the short
>>>> > version is that
>>>> >
>>>> >    1. you have to adhere to the Apache licensing requirements
>>>> >    2. you have to make it clear that your fork does not belong to the
>>>> >    Apache Flink project. (Trademarks and all that)
>>>> >
>>>> > Neither should be significant hurdles (there should also be plenty of
>>>> > online resources regarding 1), and if you do this then you can freely
>>>> share
>>>> > your fork with others.
>>>> >
>>>> > I've also pinged Martijn to take a look at this thread.
>>>> > To my knowledge the project hasn't decided anything yet.
>>>> >
>>>> > On 27/05/2023 04:05, Galen Warren wrote:
>>>> >
>>>> > Ok, I get it. No interest.
>>>> >
>>>> > If this project is being abandoned, I guess I'll work with my own
>>>> fork. Is
>>>> > there anything I should consider here? Can I share it with other
>>>> people who
>>>> > use this project?
>>>> >
>>>> > On Tue, May 16, 2023 at 10:50 AM Galen Warren <
>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
>>>> > wrote:
>>>> >
>>>> >
>>>> > Hi Martijn, since you opened this discussion thread, I'm curious what
>>>> your
>>>> > thoughts are in light of the responses? Thanks.
>>>> >
>>>> > On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <ga...@cvillewarrens.com>
>>>> <ga...@cvillewarrens.com>
>>>> > wrote:
>>>> >
>>>> >
>>>> > I use Apache Flink for stream processing, and StateFun as a hand-off
>>>> >
>>>> > point for the rest of the application.
>>>> > It serves well as a bridge between a Flink Streaming job and
>>>> > micro-services.
>>>> >
>>>> > This is essentially how I use it as well, and I would also be sad to
>>>> see
>>>> > it sunsetted. It works well; I don't know that there is a lot of new
>>>> > development required, but if there are no new Statefun releases, then
>>>> > Statefun can only be used with older Flink versions.
>>>> >
>>>> > On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
>>>> mvillalobos@kineteque.com> wrote:
>>>> >
>>>> >
>>>> > I am currently using Stateful Functions in my application.
>>>> >
>>>> > I use Apache Flink for stream processing, and StateFun as a hand-off
>>>> > point for the rest of the application.
>>>> > It serves well as a bridge between a Flink Streaming job and
>>>> > micro-services.
>>>> >
>>>> > I would be disappointed if StateFun was sunsetted.  Its a good idea.
>>>> >
>>>> > If there is anything I can do to help, as a contributor perhaps,
>>>> please
>>>> > let me know.
>>>> >
>>>> >
>>>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <ma...@apache.org>
>>>> <ma...@apache.org>
>>>> >
>>>> > wrote:
>>>> >
>>>> > Hi everyone,
>>>> >
>>>> > I want to open a discussion on the status of the Statefun Project [1]
>>>> >
>>>> > in Apache Flink. As you might have noticed, there hasn't been much
>>>> > development over the past months in the Statefun repository [2].
>>>> There is
>>>> > currently a lack of active contributors and committers who are able
>>>> to help
>>>> > with the maintenance of the project.
>>>> >
>>>> > In order to improve the situation, we need to solve the lack of
>>>> >
>>>> > committers and the lack of contributors.
>>>> >
>>>> > On the lack of committers:
>>>> >
>>>> > 1. Ideally, there are some of the current Flink committers who have
>>>> >
>>>> > the bandwidth and can help with reviewing PRs and merging them.
>>>> >
>>>> > 2. If that's not an option, it could be a consideration that current
>>>> >
>>>> > committers only approve and review PRs, that are approved by those
>>>> who are
>>>> > willing to contribute to Statefun and if the CI passes
>>>> >
>>>> > On the lack of contributors:
>>>> >
>>>> > 3. Next to having this discussion on the Dev and User mailing list, we
>>>> >
>>>> > can also create a blog with a call for new contributors on the Flink
>>>> > project website, send out some tweets on the Flink / Statefun twitter
>>>> > accounts, post messages on Slack etc. In that message, we would
>>>> inform how
>>>> > those that are interested in contributing can start and where they
>>>> could
>>>> > reach out for more information.
>>>> >
>>>> > There's also option 4. where a group of interested people would split
>>>> >
>>>> > Statefun from the Flink project and make it a separate top level
>>>> project
>>>> > under the Apache Flink umbrella (similar as recently has happened with
>>>> > Flink Table Store, which has become Apache Paimon).
>>>> >
>>>> > If we see no improvements in the coming period, we should consider
>>>> >
>>>> > sunsetting Statefun and communicate that clearly to the users.
>>>> >
>>>> > I'm looking forward to your thoughts.
>>>> >
>>>> > Best regards,
>>>> >
>>>> > Martijn
>>>> >
>>>> > [1] https://nightlies.apache.org/flink/flink-statefun-docs-master/ <
>>>> >
>>>> > https://nightlies.apache.org/flink/flink-statefun-docs-master/>
>>>> >
>>>> > [2] https://github.com/apache/flink-statefun <
>>>> >
>>>> > https://github.com/apache/flink-statefun>
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>>
>>>

Re: [DISCUSS] Status of Statefun Project

[Galen Warren via user <us...@flink.apache.org>]

Thanks.

Martijn, to answer your question, I'd need to do a small amount of work to
get a PR ready, but not much. Happy to do it if we're deciding to restart
Statefun releases -- are we?

-- Galen

On Sat, Jun 17, 2023 at 9:47 AM Tzu-Li (Gordon) Tai <tz...@apache.org>
wrote:

> > Perhaps he could weigh in on whether the combination of automated tests
> plus those smoke tests should be sufficient for testing with new Flink
> versions
>
> What we usually did at the bare minimum for new StateFun releases was the
> following:
>
>    1. Build tests (including the smoke tests in the e2e module, which
>    covers important tests like exactly-once verification)
>    2. Updating the flink-statefun-playground repo and manually running
>    all language examples there.
>
> If upgrading Flink versions was the only change in the release, I'd
> probably say that this is sufficient.
>
> Best,
> Gordon
>
> On Thu, Jun 15, 2023 at 5:25 AM Martijn Visser <ma...@apache.org>
> wrote:
>
>> Let me know if you have a PR for a Flink update :)
>>
>> On Thu, Jun 8, 2023 at 5:52 PM Galen Warren via user <
>> user@flink.apache.org> wrote:
>>
>>> Thanks Martijn.
>>>
>>> Personally, I'm already using a local fork of Statefun that is
>>> compatible with Flink 1.16.x, so I wouldn't have any need for a released
>>> version compatible with 1.15.x. I'd be happy to do the PRs to modify
>>> Statefun to work with new versions of Flink as they come along.
>>>
>>> As for testing, Statefun does have unit tests and Gordon also sent me
>>> instructions a while back for how to do some additional smoke tests which
>>> are pretty straightforward. Perhaps he could weigh in on whether the
>>> combination of automated tests plus those smoke tests should be sufficient
>>> for testing with new Flink versions (I believe the answer is yes).
>>>
>>> -- Galen
>>>
>>>
>>>
>>> On Thu, Jun 8, 2023 at 8:01 AM Martijn Visser <ma...@apache.org>
>>> wrote:
>>>
>>>> Hi all,
>>>>
>>>> Apologies for the late reply.
>>>>
>>>> I'm willing to help out with merging requests in Statefun to keep them
>>>> compatible with new Flink releases and create new releases. I do think
>>>> that
>>>> validation of the functionality of these releases depends a lot on those
>>>> who do these compatibility updates, with PMC members helping out with
>>>> the
>>>> formal process.
>>>>
>>>> > Why can't the Apache Software Foundation allow community members to
>>>> bring
>>>> it up to date?
>>>>
>>>> There's nothing preventing anyone from reviewing any of the current PRs
>>>> or
>>>> opening new ones. However, none of them are approved [1], so there's
>>>> also
>>>> nothing to merge.
>>>>
>>>> > I believe that there are people and companies on this mailing list
>>>> interested in supporting Apache Flink Stateful Functions.
>>>>
>>>> If so, then now is the time to show.
>>>>
>>>> Would there be a preference to create a release with Galen's merged
>>>> compatibility update to Flink 1.15.2, or do we want to skip that and go
>>>> straight to a newer version?
>>>>
>>>> Best regards,
>>>>
>>>> Martijn
>>>>
>>>> [1]
>>>>
>>>> https://github.com/apache/flink-statefun/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved
>>>>
>>>> On Tue, Jun 6, 2023 at 3:55 PM Marco Villalobos <
>>>> mvillalobos@kineteque.com>
>>>> wrote:
>>>>
>>>> > Why can't the Apache Software Foundation allow community members to
>>>> bring
>>>> > it up to date?
>>>> >
>>>> > What's the process for that?
>>>> >
>>>> > I believe that there are people and companies on this mailing list
>>>> > interested in supporting Apache Flink Stateful Functions.
>>>> >
>>>> > You already had two people on this thread express interest.
>>>> >
>>>> > At the very least, we could keep the library versions up to date.
>>>> >
>>>> > There are only a small list of new features that might be worthwhile:
>>>> >
>>>> > 1. event time processing
>>>> > 2. state rest api
>>>> >
>>>> >
>>>> > On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <ch...@apache.org>
>>>> wrote:
>>>> >
>>>> > If you were to fork it *and want to redistribute it* then the short
>>>> > version is that
>>>> >
>>>> >    1. you have to adhere to the Apache licensing requirements
>>>> >    2. you have to make it clear that your fork does not belong to the
>>>> >    Apache Flink project. (Trademarks and all that)
>>>> >
>>>> > Neither should be significant hurdles (there should also be plenty of
>>>> > online resources regarding 1), and if you do this then you can freely
>>>> share
>>>> > your fork with others.
>>>> >
>>>> > I've also pinged Martijn to take a look at this thread.
>>>> > To my knowledge the project hasn't decided anything yet.
>>>> >
>>>> > On 27/05/2023 04:05, Galen Warren wrote:
>>>> >
>>>> > Ok, I get it. No interest.
>>>> >
>>>> > If this project is being abandoned, I guess I'll work with my own
>>>> fork. Is
>>>> > there anything I should consider here? Can I share it with other
>>>> people who
>>>> > use this project?
>>>> >
>>>> > On Tue, May 16, 2023 at 10:50 AM Galen Warren <
>>>> galen@cvillewarrens.com> <ga...@cvillewarrens.com>
>>>> > wrote:
>>>> >
>>>> >
>>>> > Hi Martijn, since you opened this discussion thread, I'm curious what
>>>> your
>>>> > thoughts are in light of the responses? Thanks.
>>>> >
>>>> > On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <ga...@cvillewarrens.com>
>>>> <ga...@cvillewarrens.com>
>>>> > wrote:
>>>> >
>>>> >
>>>> > I use Apache Flink for stream processing, and StateFun as a hand-off
>>>> >
>>>> > point for the rest of the application.
>>>> > It serves well as a bridge between a Flink Streaming job and
>>>> > micro-services.
>>>> >
>>>> > This is essentially how I use it as well, and I would also be sad to
>>>> see
>>>> > it sunsetted. It works well; I don't know that there is a lot of new
>>>> > development required, but if there are no new Statefun releases, then
>>>> > Statefun can only be used with older Flink versions.
>>>> >
>>>> > On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
>>>> mvillalobos@kineteque.com> wrote:
>>>> >
>>>> >
>>>> > I am currently using Stateful Functions in my application.
>>>> >
>>>> > I use Apache Flink for stream processing, and StateFun as a hand-off
>>>> > point for the rest of the application.
>>>> > It serves well as a bridge between a Flink Streaming job and
>>>> > micro-services.
>>>> >
>>>> > I would be disappointed if StateFun was sunsetted.  Its a good idea.
>>>> >
>>>> > If there is anything I can do to help, as a contributor perhaps,
>>>> please
>>>> > let me know.
>>>> >
>>>> >
>>>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <ma...@apache.org>
>>>> <ma...@apache.org>
>>>> >
>>>> > wrote:
>>>> >
>>>> > Hi everyone,
>>>> >
>>>> > I want to open a discussion on the status of the Statefun Project [1]
>>>> >
>>>> > in Apache Flink. As you might have noticed, there hasn't been much
>>>> > development over the past months in the Statefun repository [2].
>>>> There is
>>>> > currently a lack of active contributors and committers who are able
>>>> to help
>>>> > with the maintenance of the project.
>>>> >
>>>> > In order to improve the situation, we need to solve the lack of
>>>> >
>>>> > committers and the lack of contributors.
>>>> >
>>>> > On the lack of committers:
>>>> >
>>>> > 1. Ideally, there are some of the current Flink committers who have
>>>> >
>>>> > the bandwidth and can help with reviewing PRs and merging them.
>>>> >
>>>> > 2. If that's not an option, it could be a consideration that current
>>>> >
>>>> > committers only approve and review PRs, that are approved by those
>>>> who are
>>>> > willing to contribute to Statefun and if the CI passes
>>>> >
>>>> > On the lack of contributors:
>>>> >
>>>> > 3. Next to having this discussion on the Dev and User mailing list, we
>>>> >
>>>> > can also create a blog with a call for new contributors on the Flink
>>>> > project website, send out some tweets on the Flink / Statefun twitter
>>>> > accounts, post messages on Slack etc. In that message, we would
>>>> inform how
>>>> > those that are interested in contributing can start and where they
>>>> could
>>>> > reach out for more information.
>>>> >
>>>> > There's also option 4. where a group of interested people would split
>>>> >
>>>> > Statefun from the Flink project and make it a separate top level
>>>> project
>>>> > under the Apache Flink umbrella (similar as recently has happened with
>>>> > Flink Table Store, which has become Apache Paimon).
>>>> >
>>>> > If we see no improvements in the coming period, we should consider
>>>> >
>>>> > sunsetting Statefun and communicate that clearly to the users.
>>>> >
>>>> > I'm looking forward to your thoughts.
>>>> >
>>>> > Best regards,
>>>> >
>>>> > Martijn
>>>> >
>>>> > [1] https://nightlies.apache.org/flink/flink-statefun-docs-master/ <
>>>> >
>>>> > https://nightlies.apache.org/flink/flink-statefun-docs-master/>
>>>> >
>>>> > [2] https://github.com/apache/flink-statefun <
>>>> >
>>>> > https://github.com/apache/flink-statefun>
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>>
>>>

Re: [DISCUSS] Status of Statefun Project

["Tzu-Li (Gordon) Tai" <tz...@apache.org>]

> Perhaps he could weigh in on whether the combination of automated tests
plus those smoke tests should be sufficient for testing with new Flink
versions

What we usually did at the bare minimum for new StateFun releases was the
following:

   1. Build tests (including the smoke tests in the e2e module, which
   covers important tests like exactly-once verification)
   2. Updating the flink-statefun-playground repo and manually running all
   language examples there.

If upgrading Flink versions was the only change in the release, I'd
probably say that this is sufficient.

Best,
Gordon

On Thu, Jun 15, 2023 at 5:25 AM Martijn Visser <ma...@apache.org>
wrote:

> Let me know if you have a PR for a Flink update :)
>
> On Thu, Jun 8, 2023 at 5:52 PM Galen Warren via user <
> user@flink.apache.org> wrote:
>
>> Thanks Martijn.
>>
>> Personally, I'm already using a local fork of Statefun that is compatible
>> with Flink 1.16.x, so I wouldn't have any need for a released version
>> compatible with 1.15.x. I'd be happy to do the PRs to modify Statefun to
>> work with new versions of Flink as they come along.
>>
>> As for testing, Statefun does have unit tests and Gordon also sent me
>> instructions a while back for how to do some additional smoke tests which
>> are pretty straightforward. Perhaps he could weigh in on whether the
>> combination of automated tests plus those smoke tests should be sufficient
>> for testing with new Flink versions (I believe the answer is yes).
>>
>> -- Galen
>>
>>
>>
>> On Thu, Jun 8, 2023 at 8:01 AM Martijn Visser <ma...@apache.org>
>> wrote:
>>
>>> Hi all,
>>>
>>> Apologies for the late reply.
>>>
>>> I'm willing to help out with merging requests in Statefun to keep them
>>> compatible with new Flink releases and create new releases. I do think
>>> that
>>> validation of the functionality of these releases depends a lot on those
>>> who do these compatibility updates, with PMC members helping out with the
>>> formal process.
>>>
>>> > Why can't the Apache Software Foundation allow community members to
>>> bring
>>> it up to date?
>>>
>>> There's nothing preventing anyone from reviewing any of the current PRs
>>> or
>>> opening new ones. However, none of them are approved [1], so there's also
>>> nothing to merge.
>>>
>>> > I believe that there are people and companies on this mailing list
>>> interested in supporting Apache Flink Stateful Functions.
>>>
>>> If so, then now is the time to show.
>>>
>>> Would there be a preference to create a release with Galen's merged
>>> compatibility update to Flink 1.15.2, or do we want to skip that and go
>>> straight to a newer version?
>>>
>>> Best regards,
>>>
>>> Martijn
>>>
>>> [1]
>>>
>>> https://github.com/apache/flink-statefun/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved
>>>
>>> On Tue, Jun 6, 2023 at 3:55 PM Marco Villalobos <
>>> mvillalobos@kineteque.com>
>>> wrote:
>>>
>>> > Why can't the Apache Software Foundation allow community members to
>>> bring
>>> > it up to date?
>>> >
>>> > What's the process for that?
>>> >
>>> > I believe that there are people and companies on this mailing list
>>> > interested in supporting Apache Flink Stateful Functions.
>>> >
>>> > You already had two people on this thread express interest.
>>> >
>>> > At the very least, we could keep the library versions up to date.
>>> >
>>> > There are only a small list of new features that might be worthwhile:
>>> >
>>> > 1. event time processing
>>> > 2. state rest api
>>> >
>>> >
>>> > On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <ch...@apache.org>
>>> wrote:
>>> >
>>> > If you were to fork it *and want to redistribute it* then the short
>>> > version is that
>>> >
>>> >    1. you have to adhere to the Apache licensing requirements
>>> >    2. you have to make it clear that your fork does not belong to the
>>> >    Apache Flink project. (Trademarks and all that)
>>> >
>>> > Neither should be significant hurdles (there should also be plenty of
>>> > online resources regarding 1), and if you do this then you can freely
>>> share
>>> > your fork with others.
>>> >
>>> > I've also pinged Martijn to take a look at this thread.
>>> > To my knowledge the project hasn't decided anything yet.
>>> >
>>> > On 27/05/2023 04:05, Galen Warren wrote:
>>> >
>>> > Ok, I get it. No interest.
>>> >
>>> > If this project is being abandoned, I guess I'll work with my own
>>> fork. Is
>>> > there anything I should consider here? Can I share it with other
>>> people who
>>> > use this project?
>>> >
>>> > On Tue, May 16, 2023 at 10:50 AM Galen Warren <ga...@cvillewarrens.com>
>>> <ga...@cvillewarrens.com>
>>> > wrote:
>>> >
>>> >
>>> > Hi Martijn, since you opened this discussion thread, I'm curious what
>>> your
>>> > thoughts are in light of the responses? Thanks.
>>> >
>>> > On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <ga...@cvillewarrens.com>
>>> <ga...@cvillewarrens.com>
>>> > wrote:
>>> >
>>> >
>>> > I use Apache Flink for stream processing, and StateFun as a hand-off
>>> >
>>> > point for the rest of the application.
>>> > It serves well as a bridge between a Flink Streaming job and
>>> > micro-services.
>>> >
>>> > This is essentially how I use it as well, and I would also be sad to
>>> see
>>> > it sunsetted. It works well; I don't know that there is a lot of new
>>> > development required, but if there are no new Statefun releases, then
>>> > Statefun can only be used with older Flink versions.
>>> >
>>> > On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
>>> mvillalobos@kineteque.com> wrote:
>>> >
>>> >
>>> > I am currently using Stateful Functions in my application.
>>> >
>>> > I use Apache Flink for stream processing, and StateFun as a hand-off
>>> > point for the rest of the application.
>>> > It serves well as a bridge between a Flink Streaming job and
>>> > micro-services.
>>> >
>>> > I would be disappointed if StateFun was sunsetted.  Its a good idea.
>>> >
>>> > If there is anything I can do to help, as a contributor perhaps, please
>>> > let me know.
>>> >
>>> >
>>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <ma...@apache.org>
>>> <ma...@apache.org>
>>> >
>>> > wrote:
>>> >
>>> > Hi everyone,
>>> >
>>> > I want to open a discussion on the status of the Statefun Project [1]
>>> >
>>> > in Apache Flink. As you might have noticed, there hasn't been much
>>> > development over the past months in the Statefun repository [2]. There
>>> is
>>> > currently a lack of active contributors and committers who are able to
>>> help
>>> > with the maintenance of the project.
>>> >
>>> > In order to improve the situation, we need to solve the lack of
>>> >
>>> > committers and the lack of contributors.
>>> >
>>> > On the lack of committers:
>>> >
>>> > 1. Ideally, there are some of the current Flink committers who have
>>> >
>>> > the bandwidth and can help with reviewing PRs and merging them.
>>> >
>>> > 2. If that's not an option, it could be a consideration that current
>>> >
>>> > committers only approve and review PRs, that are approved by those who
>>> are
>>> > willing to contribute to Statefun and if the CI passes
>>> >
>>> > On the lack of contributors:
>>> >
>>> > 3. Next to having this discussion on the Dev and User mailing list, we
>>> >
>>> > can also create a blog with a call for new contributors on the Flink
>>> > project website, send out some tweets on the Flink / Statefun twitter
>>> > accounts, post messages on Slack etc. In that message, we would inform
>>> how
>>> > those that are interested in contributing can start and where they
>>> could
>>> > reach out for more information.
>>> >
>>> > There's also option 4. where a group of interested people would split
>>> >
>>> > Statefun from the Flink project and make it a separate top level
>>> project
>>> > under the Apache Flink umbrella (similar as recently has happened with
>>> > Flink Table Store, which has become Apache Paimon).
>>> >
>>> > If we see no improvements in the coming period, we should consider
>>> >
>>> > sunsetting Statefun and communicate that clearly to the users.
>>> >
>>> > I'm looking forward to your thoughts.
>>> >
>>> > Best regards,
>>> >
>>> > Martijn
>>> >
>>> > [1] https://nightlies.apache.org/flink/flink-statefun-docs-master/ <
>>> >
>>> > https://nightlies.apache.org/flink/flink-statefun-docs-master/>
>>> >
>>> > [2] https://github.com/apache/flink-statefun <
>>> >
>>> > https://github.com/apache/flink-statefun>
>>> >
>>> >
>>> >
>>> >
>>> >
>>>
>>

Re: [DISCUSS] Status of Statefun Project

["Tzu-Li (Gordon) Tai" <tz...@apache.org>]

> Perhaps he could weigh in on whether the combination of automated tests
plus those smoke tests should be sufficient for testing with new Flink
versions

What we usually did at the bare minimum for new StateFun releases was the
following:

   1. Build tests (including the smoke tests in the e2e module, which
   covers important tests like exactly-once verification)
   2. Updating the flink-statefun-playground repo and manually running all
   language examples there.

If upgrading Flink versions was the only change in the release, I'd
probably say that this is sufficient.

Best,
Gordon

On Thu, Jun 15, 2023 at 5:25 AM Martijn Visser <ma...@apache.org>
wrote:

> Let me know if you have a PR for a Flink update :)
>
> On Thu, Jun 8, 2023 at 5:52 PM Galen Warren via user <
> user@flink.apache.org> wrote:
>
>> Thanks Martijn.
>>
>> Personally, I'm already using a local fork of Statefun that is compatible
>> with Flink 1.16.x, so I wouldn't have any need for a released version
>> compatible with 1.15.x. I'd be happy to do the PRs to modify Statefun to
>> work with new versions of Flink as they come along.
>>
>> As for testing, Statefun does have unit tests and Gordon also sent me
>> instructions a while back for how to do some additional smoke tests which
>> are pretty straightforward. Perhaps he could weigh in on whether the
>> combination of automated tests plus those smoke tests should be sufficient
>> for testing with new Flink versions (I believe the answer is yes).
>>
>> -- Galen
>>
>>
>>
>> On Thu, Jun 8, 2023 at 8:01 AM Martijn Visser <ma...@apache.org>
>> wrote:
>>
>>> Hi all,
>>>
>>> Apologies for the late reply.
>>>
>>> I'm willing to help out with merging requests in Statefun to keep them
>>> compatible with new Flink releases and create new releases. I do think
>>> that
>>> validation of the functionality of these releases depends a lot on those
>>> who do these compatibility updates, with PMC members helping out with the
>>> formal process.
>>>
>>> > Why can't the Apache Software Foundation allow community members to
>>> bring
>>> it up to date?
>>>
>>> There's nothing preventing anyone from reviewing any of the current PRs
>>> or
>>> opening new ones. However, none of them are approved [1], so there's also
>>> nothing to merge.
>>>
>>> > I believe that there are people and companies on this mailing list
>>> interested in supporting Apache Flink Stateful Functions.
>>>
>>> If so, then now is the time to show.
>>>
>>> Would there be a preference to create a release with Galen's merged
>>> compatibility update to Flink 1.15.2, or do we want to skip that and go
>>> straight to a newer version?
>>>
>>> Best regards,
>>>
>>> Martijn
>>>
>>> [1]
>>>
>>> https://github.com/apache/flink-statefun/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved
>>>
>>> On Tue, Jun 6, 2023 at 3:55 PM Marco Villalobos <
>>> mvillalobos@kineteque.com>
>>> wrote:
>>>
>>> > Why can't the Apache Software Foundation allow community members to
>>> bring
>>> > it up to date?
>>> >
>>> > What's the process for that?
>>> >
>>> > I believe that there are people and companies on this mailing list
>>> > interested in supporting Apache Flink Stateful Functions.
>>> >
>>> > You already had two people on this thread express interest.
>>> >
>>> > At the very least, we could keep the library versions up to date.
>>> >
>>> > There are only a small list of new features that might be worthwhile:
>>> >
>>> > 1. event time processing
>>> > 2. state rest api
>>> >
>>> >
>>> > On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <ch...@apache.org>
>>> wrote:
>>> >
>>> > If you were to fork it *and want to redistribute it* then the short
>>> > version is that
>>> >
>>> >    1. you have to adhere to the Apache licensing requirements
>>> >    2. you have to make it clear that your fork does not belong to the
>>> >    Apache Flink project. (Trademarks and all that)
>>> >
>>> > Neither should be significant hurdles (there should also be plenty of
>>> > online resources regarding 1), and if you do this then you can freely
>>> share
>>> > your fork with others.
>>> >
>>> > I've also pinged Martijn to take a look at this thread.
>>> > To my knowledge the project hasn't decided anything yet.
>>> >
>>> > On 27/05/2023 04:05, Galen Warren wrote:
>>> >
>>> > Ok, I get it. No interest.
>>> >
>>> > If this project is being abandoned, I guess I'll work with my own
>>> fork. Is
>>> > there anything I should consider here? Can I share it with other
>>> people who
>>> > use this project?
>>> >
>>> > On Tue, May 16, 2023 at 10:50 AM Galen Warren <ga...@cvillewarrens.com>
>>> <ga...@cvillewarrens.com>
>>> > wrote:
>>> >
>>> >
>>> > Hi Martijn, since you opened this discussion thread, I'm curious what
>>> your
>>> > thoughts are in light of the responses? Thanks.
>>> >
>>> > On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <ga...@cvillewarrens.com>
>>> <ga...@cvillewarrens.com>
>>> > wrote:
>>> >
>>> >
>>> > I use Apache Flink for stream processing, and StateFun as a hand-off
>>> >
>>> > point for the rest of the application.
>>> > It serves well as a bridge between a Flink Streaming job and
>>> > micro-services.
>>> >
>>> > This is essentially how I use it as well, and I would also be sad to
>>> see
>>> > it sunsetted. It works well; I don't know that there is a lot of new
>>> > development required, but if there are no new Statefun releases, then
>>> > Statefun can only be used with older Flink versions.
>>> >
>>> > On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
>>> mvillalobos@kineteque.com> wrote:
>>> >
>>> >
>>> > I am currently using Stateful Functions in my application.
>>> >
>>> > I use Apache Flink for stream processing, and StateFun as a hand-off
>>> > point for the rest of the application.
>>> > It serves well as a bridge between a Flink Streaming job and
>>> > micro-services.
>>> >
>>> > I would be disappointed if StateFun was sunsetted.  Its a good idea.
>>> >
>>> > If there is anything I can do to help, as a contributor perhaps, please
>>> > let me know.
>>> >
>>> >
>>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <ma...@apache.org>
>>> <ma...@apache.org>
>>> >
>>> > wrote:
>>> >
>>> > Hi everyone,
>>> >
>>> > I want to open a discussion on the status of the Statefun Project [1]
>>> >
>>> > in Apache Flink. As you might have noticed, there hasn't been much
>>> > development over the past months in the Statefun repository [2]. There
>>> is
>>> > currently a lack of active contributors and committers who are able to
>>> help
>>> > with the maintenance of the project.
>>> >
>>> > In order to improve the situation, we need to solve the lack of
>>> >
>>> > committers and the lack of contributors.
>>> >
>>> > On the lack of committers:
>>> >
>>> > 1. Ideally, there are some of the current Flink committers who have
>>> >
>>> > the bandwidth and can help with reviewing PRs and merging them.
>>> >
>>> > 2. If that's not an option, it could be a consideration that current
>>> >
>>> > committers only approve and review PRs, that are approved by those who
>>> are
>>> > willing to contribute to Statefun and if the CI passes
>>> >
>>> > On the lack of contributors:
>>> >
>>> > 3. Next to having this discussion on the Dev and User mailing list, we
>>> >
>>> > can also create a blog with a call for new contributors on the Flink
>>> > project website, send out some tweets on the Flink / Statefun twitter
>>> > accounts, post messages on Slack etc. In that message, we would inform
>>> how
>>> > those that are interested in contributing can start and where they
>>> could
>>> > reach out for more information.
>>> >
>>> > There's also option 4. where a group of interested people would split
>>> >
>>> > Statefun from the Flink project and make it a separate top level
>>> project
>>> > under the Apache Flink umbrella (similar as recently has happened with
>>> > Flink Table Store, which has become Apache Paimon).
>>> >
>>> > If we see no improvements in the coming period, we should consider
>>> >
>>> > sunsetting Statefun and communicate that clearly to the users.
>>> >
>>> > I'm looking forward to your thoughts.
>>> >
>>> > Best regards,
>>> >
>>> > Martijn
>>> >
>>> > [1] https://nightlies.apache.org/flink/flink-statefun-docs-master/ <
>>> >
>>> > https://nightlies.apache.org/flink/flink-statefun-docs-master/>
>>> >
>>> > [2] https://github.com/apache/flink-statefun <
>>> >
>>> > https://github.com/apache/flink-statefun>
>>> >
>>> >
>>> >
>>> >
>>> >
>>>
>>

Re: [DISCUSS] Status of Statefun Project

[Martijn Visser <ma...@apache.org>]

Let me know if you have a PR for a Flink update :)

On Thu, Jun 8, 2023 at 5:52 PM Galen Warren via user <us...@flink.apache.org>
wrote:

> Thanks Martijn.
>
> Personally, I'm already using a local fork of Statefun that is compatible
> with Flink 1.16.x, so I wouldn't have any need for a released version
> compatible with 1.15.x. I'd be happy to do the PRs to modify Statefun to
> work with new versions of Flink as they come along.
>
> As for testing, Statefun does have unit tests and Gordon also sent me
> instructions a while back for how to do some additional smoke tests which
> are pretty straightforward. Perhaps he could weigh in on whether the
> combination of automated tests plus those smoke tests should be sufficient
> for testing with new Flink versions (I believe the answer is yes).
>
> -- Galen
>
>
>
> On Thu, Jun 8, 2023 at 8:01 AM Martijn Visser <ma...@apache.org>
> wrote:
>
>> Hi all,
>>
>> Apologies for the late reply.
>>
>> I'm willing to help out with merging requests in Statefun to keep them
>> compatible with new Flink releases and create new releases. I do think
>> that
>> validation of the functionality of these releases depends a lot on those
>> who do these compatibility updates, with PMC members helping out with the
>> formal process.
>>
>> > Why can't the Apache Software Foundation allow community members to
>> bring
>> it up to date?
>>
>> There's nothing preventing anyone from reviewing any of the current PRs or
>> opening new ones. However, none of them are approved [1], so there's also
>> nothing to merge.
>>
>> > I believe that there are people and companies on this mailing list
>> interested in supporting Apache Flink Stateful Functions.
>>
>> If so, then now is the time to show.
>>
>> Would there be a preference to create a release with Galen's merged
>> compatibility update to Flink 1.15.2, or do we want to skip that and go
>> straight to a newer version?
>>
>> Best regards,
>>
>> Martijn
>>
>> [1]
>>
>> https://github.com/apache/flink-statefun/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved
>>
>> On Tue, Jun 6, 2023 at 3:55 PM Marco Villalobos <
>> mvillalobos@kineteque.com>
>> wrote:
>>
>> > Why can't the Apache Software Foundation allow community members to
>> bring
>> > it up to date?
>> >
>> > What's the process for that?
>> >
>> > I believe that there are people and companies on this mailing list
>> > interested in supporting Apache Flink Stateful Functions.
>> >
>> > You already had two people on this thread express interest.
>> >
>> > At the very least, we could keep the library versions up to date.
>> >
>> > There are only a small list of new features that might be worthwhile:
>> >
>> > 1. event time processing
>> > 2. state rest api
>> >
>> >
>> > On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <ch...@apache.org>
>> wrote:
>> >
>> > If you were to fork it *and want to redistribute it* then the short
>> > version is that
>> >
>> >    1. you have to adhere to the Apache licensing requirements
>> >    2. you have to make it clear that your fork does not belong to the
>> >    Apache Flink project. (Trademarks and all that)
>> >
>> > Neither should be significant hurdles (there should also be plenty of
>> > online resources regarding 1), and if you do this then you can freely
>> share
>> > your fork with others.
>> >
>> > I've also pinged Martijn to take a look at this thread.
>> > To my knowledge the project hasn't decided anything yet.
>> >
>> > On 27/05/2023 04:05, Galen Warren wrote:
>> >
>> > Ok, I get it. No interest.
>> >
>> > If this project is being abandoned, I guess I'll work with my own fork.
>> Is
>> > there anything I should consider here? Can I share it with other people
>> who
>> > use this project?
>> >
>> > On Tue, May 16, 2023 at 10:50 AM Galen Warren <ga...@cvillewarrens.com>
>> <ga...@cvillewarrens.com>
>> > wrote:
>> >
>> >
>> > Hi Martijn, since you opened this discussion thread, I'm curious what
>> your
>> > thoughts are in light of the responses? Thanks.
>> >
>> > On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <ga...@cvillewarrens.com>
>> <ga...@cvillewarrens.com>
>> > wrote:
>> >
>> >
>> > I use Apache Flink for stream processing, and StateFun as a hand-off
>> >
>> > point for the rest of the application.
>> > It serves well as a bridge between a Flink Streaming job and
>> > micro-services.
>> >
>> > This is essentially how I use it as well, and I would also be sad to see
>> > it sunsetted. It works well; I don't know that there is a lot of new
>> > development required, but if there are no new Statefun releases, then
>> > Statefun can only be used with older Flink versions.
>> >
>> > On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
>> mvillalobos@kineteque.com> wrote:
>> >
>> >
>> > I am currently using Stateful Functions in my application.
>> >
>> > I use Apache Flink for stream processing, and StateFun as a hand-off
>> > point for the rest of the application.
>> > It serves well as a bridge between a Flink Streaming job and
>> > micro-services.
>> >
>> > I would be disappointed if StateFun was sunsetted.  Its a good idea.
>> >
>> > If there is anything I can do to help, as a contributor perhaps, please
>> > let me know.
>> >
>> >
>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <ma...@apache.org> <
>> martijnvisser@apache.org>
>> >
>> > wrote:
>> >
>> > Hi everyone,
>> >
>> > I want to open a discussion on the status of the Statefun Project [1]
>> >
>> > in Apache Flink. As you might have noticed, there hasn't been much
>> > development over the past months in the Statefun repository [2]. There
>> is
>> > currently a lack of active contributors and committers who are able to
>> help
>> > with the maintenance of the project.
>> >
>> > In order to improve the situation, we need to solve the lack of
>> >
>> > committers and the lack of contributors.
>> >
>> > On the lack of committers:
>> >
>> > 1. Ideally, there are some of the current Flink committers who have
>> >
>> > the bandwidth and can help with reviewing PRs and merging them.
>> >
>> > 2. If that's not an option, it could be a consideration that current
>> >
>> > committers only approve and review PRs, that are approved by those who
>> are
>> > willing to contribute to Statefun and if the CI passes
>> >
>> > On the lack of contributors:
>> >
>> > 3. Next to having this discussion on the Dev and User mailing list, we
>> >
>> > can also create a blog with a call for new contributors on the Flink
>> > project website, send out some tweets on the Flink / Statefun twitter
>> > accounts, post messages on Slack etc. In that message, we would inform
>> how
>> > those that are interested in contributing can start and where they could
>> > reach out for more information.
>> >
>> > There's also option 4. where a group of interested people would split
>> >
>> > Statefun from the Flink project and make it a separate top level project
>> > under the Apache Flink umbrella (similar as recently has happened with
>> > Flink Table Store, which has become Apache Paimon).
>> >
>> > If we see no improvements in the coming period, we should consider
>> >
>> > sunsetting Statefun and communicate that clearly to the users.
>> >
>> > I'm looking forward to your thoughts.
>> >
>> > Best regards,
>> >
>> > Martijn
>> >
>> > [1] https://nightlies.apache.org/flink/flink-statefun-docs-master/ <
>> >
>> > https://nightlies.apache.org/flink/flink-statefun-docs-master/>
>> >
>> > [2] https://github.com/apache/flink-statefun <
>> >
>> > https://github.com/apache/flink-statefun>
>> >
>> >
>> >
>> >
>> >
>>
>

Re: [DISCUSS] Status of Statefun Project

[Martijn Visser <ma...@apache.org>]

Let me know if you have a PR for a Flink update :)

On Thu, Jun 8, 2023 at 5:52 PM Galen Warren via user <us...@flink.apache.org>
wrote:

> Thanks Martijn.
>
> Personally, I'm already using a local fork of Statefun that is compatible
> with Flink 1.16.x, so I wouldn't have any need for a released version
> compatible with 1.15.x. I'd be happy to do the PRs to modify Statefun to
> work with new versions of Flink as they come along.
>
> As for testing, Statefun does have unit tests and Gordon also sent me
> instructions a while back for how to do some additional smoke tests which
> are pretty straightforward. Perhaps he could weigh in on whether the
> combination of automated tests plus those smoke tests should be sufficient
> for testing with new Flink versions (I believe the answer is yes).
>
> -- Galen
>
>
>
> On Thu, Jun 8, 2023 at 8:01 AM Martijn Visser <ma...@apache.org>
> wrote:
>
>> Hi all,
>>
>> Apologies for the late reply.
>>
>> I'm willing to help out with merging requests in Statefun to keep them
>> compatible with new Flink releases and create new releases. I do think
>> that
>> validation of the functionality of these releases depends a lot on those
>> who do these compatibility updates, with PMC members helping out with the
>> formal process.
>>
>> > Why can't the Apache Software Foundation allow community members to
>> bring
>> it up to date?
>>
>> There's nothing preventing anyone from reviewing any of the current PRs or
>> opening new ones. However, none of them are approved [1], so there's also
>> nothing to merge.
>>
>> > I believe that there are people and companies on this mailing list
>> interested in supporting Apache Flink Stateful Functions.
>>
>> If so, then now is the time to show.
>>
>> Would there be a preference to create a release with Galen's merged
>> compatibility update to Flink 1.15.2, or do we want to skip that and go
>> straight to a newer version?
>>
>> Best regards,
>>
>> Martijn
>>
>> [1]
>>
>> https://github.com/apache/flink-statefun/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved
>>
>> On Tue, Jun 6, 2023 at 3:55 PM Marco Villalobos <
>> mvillalobos@kineteque.com>
>> wrote:
>>
>> > Why can't the Apache Software Foundation allow community members to
>> bring
>> > it up to date?
>> >
>> > What's the process for that?
>> >
>> > I believe that there are people and companies on this mailing list
>> > interested in supporting Apache Flink Stateful Functions.
>> >
>> > You already had two people on this thread express interest.
>> >
>> > At the very least, we could keep the library versions up to date.
>> >
>> > There are only a small list of new features that might be worthwhile:
>> >
>> > 1. event time processing
>> > 2. state rest api
>> >
>> >
>> > On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <ch...@apache.org>
>> wrote:
>> >
>> > If you were to fork it *and want to redistribute it* then the short
>> > version is that
>> >
>> >    1. you have to adhere to the Apache licensing requirements
>> >    2. you have to make it clear that your fork does not belong to the
>> >    Apache Flink project. (Trademarks and all that)
>> >
>> > Neither should be significant hurdles (there should also be plenty of
>> > online resources regarding 1), and if you do this then you can freely
>> share
>> > your fork with others.
>> >
>> > I've also pinged Martijn to take a look at this thread.
>> > To my knowledge the project hasn't decided anything yet.
>> >
>> > On 27/05/2023 04:05, Galen Warren wrote:
>> >
>> > Ok, I get it. No interest.
>> >
>> > If this project is being abandoned, I guess I'll work with my own fork.
>> Is
>> > there anything I should consider here? Can I share it with other people
>> who
>> > use this project?
>> >
>> > On Tue, May 16, 2023 at 10:50 AM Galen Warren <ga...@cvillewarrens.com>
>> <ga...@cvillewarrens.com>
>> > wrote:
>> >
>> >
>> > Hi Martijn, since you opened this discussion thread, I'm curious what
>> your
>> > thoughts are in light of the responses? Thanks.
>> >
>> > On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <ga...@cvillewarrens.com>
>> <ga...@cvillewarrens.com>
>> > wrote:
>> >
>> >
>> > I use Apache Flink for stream processing, and StateFun as a hand-off
>> >
>> > point for the rest of the application.
>> > It serves well as a bridge between a Flink Streaming job and
>> > micro-services.
>> >
>> > This is essentially how I use it as well, and I would also be sad to see
>> > it sunsetted. It works well; I don't know that there is a lot of new
>> > development required, but if there are no new Statefun releases, then
>> > Statefun can only be used with older Flink versions.
>> >
>> > On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
>> mvillalobos@kineteque.com> wrote:
>> >
>> >
>> > I am currently using Stateful Functions in my application.
>> >
>> > I use Apache Flink for stream processing, and StateFun as a hand-off
>> > point for the rest of the application.
>> > It serves well as a bridge between a Flink Streaming job and
>> > micro-services.
>> >
>> > I would be disappointed if StateFun was sunsetted.  Its a good idea.
>> >
>> > If there is anything I can do to help, as a contributor perhaps, please
>> > let me know.
>> >
>> >
>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <ma...@apache.org> <
>> martijnvisser@apache.org>
>> >
>> > wrote:
>> >
>> > Hi everyone,
>> >
>> > I want to open a discussion on the status of the Statefun Project [1]
>> >
>> > in Apache Flink. As you might have noticed, there hasn't been much
>> > development over the past months in the Statefun repository [2]. There
>> is
>> > currently a lack of active contributors and committers who are able to
>> help
>> > with the maintenance of the project.
>> >
>> > In order to improve the situation, we need to solve the lack of
>> >
>> > committers and the lack of contributors.
>> >
>> > On the lack of committers:
>> >
>> > 1. Ideally, there are some of the current Flink committers who have
>> >
>> > the bandwidth and can help with reviewing PRs and merging them.
>> >
>> > 2. If that's not an option, it could be a consideration that current
>> >
>> > committers only approve and review PRs, that are approved by those who
>> are
>> > willing to contribute to Statefun and if the CI passes
>> >
>> > On the lack of contributors:
>> >
>> > 3. Next to having this discussion on the Dev and User mailing list, we
>> >
>> > can also create a blog with a call for new contributors on the Flink
>> > project website, send out some tweets on the Flink / Statefun twitter
>> > accounts, post messages on Slack etc. In that message, we would inform
>> how
>> > those that are interested in contributing can start and where they could
>> > reach out for more information.
>> >
>> > There's also option 4. where a group of interested people would split
>> >
>> > Statefun from the Flink project and make it a separate top level project
>> > under the Apache Flink umbrella (similar as recently has happened with
>> > Flink Table Store, which has become Apache Paimon).
>> >
>> > If we see no improvements in the coming period, we should consider
>> >
>> > sunsetting Statefun and communicate that clearly to the users.
>> >
>> > I'm looking forward to your thoughts.
>> >
>> > Best regards,
>> >
>> > Martijn
>> >
>> > [1] https://nightlies.apache.org/flink/flink-statefun-docs-master/ <
>> >
>> > https://nightlies.apache.org/flink/flink-statefun-docs-master/>
>> >
>> > [2] https://github.com/apache/flink-statefun <
>> >
>> > https://github.com/apache/flink-statefun>
>> >
>> >
>> >
>> >
>> >
>>
>

Re: [DISCUSS] Status of Statefun Project

[Galen Warren via user <us...@flink.apache.org>]

Thanks Martijn.

Personally, I'm already using a local fork of Statefun that is compatible
with Flink 1.16.x, so I wouldn't have any need for a released version
compatible with 1.15.x. I'd be happy to do the PRs to modify Statefun to
work with new versions of Flink as they come along.

As for testing, Statefun does have unit tests and Gordon also sent me
instructions a while back for how to do some additional smoke tests which
are pretty straightforward. Perhaps he could weigh in on whether the
combination of automated tests plus those smoke tests should be sufficient
for testing with new Flink versions (I believe the answer is yes).

-- Galen



On Thu, Jun 8, 2023 at 8:01 AM Martijn Visser <ma...@apache.org>
wrote:

> Hi all,
>
> Apologies for the late reply.
>
> I'm willing to help out with merging requests in Statefun to keep them
> compatible with new Flink releases and create new releases. I do think that
> validation of the functionality of these releases depends a lot on those
> who do these compatibility updates, with PMC members helping out with the
> formal process.
>
> > Why can't the Apache Software Foundation allow community members to bring
> it up to date?
>
> There's nothing preventing anyone from reviewing any of the current PRs or
> opening new ones. However, none of them are approved [1], so there's also
> nothing to merge.
>
> > I believe that there are people and companies on this mailing list
> interested in supporting Apache Flink Stateful Functions.
>
> If so, then now is the time to show.
>
> Would there be a preference to create a release with Galen's merged
> compatibility update to Flink 1.15.2, or do we want to skip that and go
> straight to a newer version?
>
> Best regards,
>
> Martijn
>
> [1]
>
> https://github.com/apache/flink-statefun/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved
>
> On Tue, Jun 6, 2023 at 3:55 PM Marco Villalobos <mvillalobos@kineteque.com
> >
> wrote:
>
> > Why can't the Apache Software Foundation allow community members to bring
> > it up to date?
> >
> > What's the process for that?
> >
> > I believe that there are people and companies on this mailing list
> > interested in supporting Apache Flink Stateful Functions.
> >
> > You already had two people on this thread express interest.
> >
> > At the very least, we could keep the library versions up to date.
> >
> > There are only a small list of new features that might be worthwhile:
> >
> > 1. event time processing
> > 2. state rest api
> >
> >
> > On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <ch...@apache.org> wrote:
> >
> > If you were to fork it *and want to redistribute it* then the short
> > version is that
> >
> >    1. you have to adhere to the Apache licensing requirements
> >    2. you have to make it clear that your fork does not belong to the
> >    Apache Flink project. (Trademarks and all that)
> >
> > Neither should be significant hurdles (there should also be plenty of
> > online resources regarding 1), and if you do this then you can freely
> share
> > your fork with others.
> >
> > I've also pinged Martijn to take a look at this thread.
> > To my knowledge the project hasn't decided anything yet.
> >
> > On 27/05/2023 04:05, Galen Warren wrote:
> >
> > Ok, I get it. No interest.
> >
> > If this project is being abandoned, I guess I'll work with my own fork.
> Is
> > there anything I should consider here? Can I share it with other people
> who
> > use this project?
> >
> > On Tue, May 16, 2023 at 10:50 AM Galen Warren <ga...@cvillewarrens.com>
> <ga...@cvillewarrens.com>
> > wrote:
> >
> >
> > Hi Martijn, since you opened this discussion thread, I'm curious what
> your
> > thoughts are in light of the responses? Thanks.
> >
> > On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <ga...@cvillewarrens.com> <
> galen@cvillewarrens.com>
> > wrote:
> >
> >
> > I use Apache Flink for stream processing, and StateFun as a hand-off
> >
> > point for the rest of the application.
> > It serves well as a bridge between a Flink Streaming job and
> > micro-services.
> >
> > This is essentially how I use it as well, and I would also be sad to see
> > it sunsetted. It works well; I don't know that there is a lot of new
> > development required, but if there are no new Statefun releases, then
> > Statefun can only be used with older Flink versions.
> >
> > On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
> mvillalobos@kineteque.com> wrote:
> >
> >
> > I am currently using Stateful Functions in my application.
> >
> > I use Apache Flink for stream processing, and StateFun as a hand-off
> > point for the rest of the application.
> > It serves well as a bridge between a Flink Streaming job and
> > micro-services.
> >
> > I would be disappointed if StateFun was sunsetted.  Its a good idea.
> >
> > If there is anything I can do to help, as a contributor perhaps, please
> > let me know.
> >
> >
> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <ma...@apache.org> <
> martijnvisser@apache.org>
> >
> > wrote:
> >
> > Hi everyone,
> >
> > I want to open a discussion on the status of the Statefun Project [1]
> >
> > in Apache Flink. As you might have noticed, there hasn't been much
> > development over the past months in the Statefun repository [2]. There is
> > currently a lack of active contributors and committers who are able to
> help
> > with the maintenance of the project.
> >
> > In order to improve the situation, we need to solve the lack of
> >
> > committers and the lack of contributors.
> >
> > On the lack of committers:
> >
> > 1. Ideally, there are some of the current Flink committers who have
> >
> > the bandwidth and can help with reviewing PRs and merging them.
> >
> > 2. If that's not an option, it could be a consideration that current
> >
> > committers only approve and review PRs, that are approved by those who
> are
> > willing to contribute to Statefun and if the CI passes
> >
> > On the lack of contributors:
> >
> > 3. Next to having this discussion on the Dev and User mailing list, we
> >
> > can also create a blog with a call for new contributors on the Flink
> > project website, send out some tweets on the Flink / Statefun twitter
> > accounts, post messages on Slack etc. In that message, we would inform
> how
> > those that are interested in contributing can start and where they could
> > reach out for more information.
> >
> > There's also option 4. where a group of interested people would split
> >
> > Statefun from the Flink project and make it a separate top level project
> > under the Apache Flink umbrella (similar as recently has happened with
> > Flink Table Store, which has become Apache Paimon).
> >
> > If we see no improvements in the coming period, we should consider
> >
> > sunsetting Statefun and communicate that clearly to the users.
> >
> > I'm looking forward to your thoughts.
> >
> > Best regards,
> >
> > Martijn
> >
> > [1] https://nightlies.apache.org/flink/flink-statefun-docs-master/ <
> >
> > https://nightlies.apache.org/flink/flink-statefun-docs-master/>
> >
> > [2] https://github.com/apache/flink-statefun <
> >
> > https://github.com/apache/flink-statefun>
> >
> >
> >
> >
> >
>

Re: [DISCUSS] Status of Statefun Project

[Galen Warren <ga...@cvillewarrens.com.INVALID>]

Thanks Martijn.

Personally, I'm already using a local fork of Statefun that is compatible
with Flink 1.16.x, so I wouldn't have any need for a released version
compatible with 1.15.x. I'd be happy to do the PRs to modify Statefun to
work with new versions of Flink as they come along.

As for testing, Statefun does have unit tests and Gordon also sent me
instructions a while back for how to do some additional smoke tests which
are pretty straightforward. Perhaps he could weigh in on whether the
combination of automated tests plus those smoke tests should be sufficient
for testing with new Flink versions (I believe the answer is yes).

-- Galen



On Thu, Jun 8, 2023 at 8:01 AM Martijn Visser <ma...@apache.org>
wrote:

> Hi all,
>
> Apologies for the late reply.
>
> I'm willing to help out with merging requests in Statefun to keep them
> compatible with new Flink releases and create new releases. I do think that
> validation of the functionality of these releases depends a lot on those
> who do these compatibility updates, with PMC members helping out with the
> formal process.
>
> > Why can't the Apache Software Foundation allow community members to bring
> it up to date?
>
> There's nothing preventing anyone from reviewing any of the current PRs or
> opening new ones. However, none of them are approved [1], so there's also
> nothing to merge.
>
> > I believe that there are people and companies on this mailing list
> interested in supporting Apache Flink Stateful Functions.
>
> If so, then now is the time to show.
>
> Would there be a preference to create a release with Galen's merged
> compatibility update to Flink 1.15.2, or do we want to skip that and go
> straight to a newer version?
>
> Best regards,
>
> Martijn
>
> [1]
>
> https://github.com/apache/flink-statefun/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved
>
> On Tue, Jun 6, 2023 at 3:55 PM Marco Villalobos <mvillalobos@kineteque.com
> >
> wrote:
>
> > Why can't the Apache Software Foundation allow community members to bring
> > it up to date?
> >
> > What's the process for that?
> >
> > I believe that there are people and companies on this mailing list
> > interested in supporting Apache Flink Stateful Functions.
> >
> > You already had two people on this thread express interest.
> >
> > At the very least, we could keep the library versions up to date.
> >
> > There are only a small list of new features that might be worthwhile:
> >
> > 1. event time processing
> > 2. state rest api
> >
> >
> > On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <ch...@apache.org> wrote:
> >
> > If you were to fork it *and want to redistribute it* then the short
> > version is that
> >
> >    1. you have to adhere to the Apache licensing requirements
> >    2. you have to make it clear that your fork does not belong to the
> >    Apache Flink project. (Trademarks and all that)
> >
> > Neither should be significant hurdles (there should also be plenty of
> > online resources regarding 1), and if you do this then you can freely
> share
> > your fork with others.
> >
> > I've also pinged Martijn to take a look at this thread.
> > To my knowledge the project hasn't decided anything yet.
> >
> > On 27/05/2023 04:05, Galen Warren wrote:
> >
> > Ok, I get it. No interest.
> >
> > If this project is being abandoned, I guess I'll work with my own fork.
> Is
> > there anything I should consider here? Can I share it with other people
> who
> > use this project?
> >
> > On Tue, May 16, 2023 at 10:50 AM Galen Warren <ga...@cvillewarrens.com>
> <ga...@cvillewarrens.com>
> > wrote:
> >
> >
> > Hi Martijn, since you opened this discussion thread, I'm curious what
> your
> > thoughts are in light of the responses? Thanks.
> >
> > On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <ga...@cvillewarrens.com> <
> galen@cvillewarrens.com>
> > wrote:
> >
> >
> > I use Apache Flink for stream processing, and StateFun as a hand-off
> >
> > point for the rest of the application.
> > It serves well as a bridge between a Flink Streaming job and
> > micro-services.
> >
> > This is essentially how I use it as well, and I would also be sad to see
> > it sunsetted. It works well; I don't know that there is a lot of new
> > development required, but if there are no new Statefun releases, then
> > Statefun can only be used with older Flink versions.
> >
> > On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
> mvillalobos@kineteque.com> wrote:
> >
> >
> > I am currently using Stateful Functions in my application.
> >
> > I use Apache Flink for stream processing, and StateFun as a hand-off
> > point for the rest of the application.
> > It serves well as a bridge between a Flink Streaming job and
> > micro-services.
> >
> > I would be disappointed if StateFun was sunsetted.  Its a good idea.
> >
> > If there is anything I can do to help, as a contributor perhaps, please
> > let me know.
> >
> >
> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <ma...@apache.org> <
> martijnvisser@apache.org>
> >
> > wrote:
> >
> > Hi everyone,
> >
> > I want to open a discussion on the status of the Statefun Project [1]
> >
> > in Apache Flink. As you might have noticed, there hasn't been much
> > development over the past months in the Statefun repository [2]. There is
> > currently a lack of active contributors and committers who are able to
> help
> > with the maintenance of the project.
> >
> > In order to improve the situation, we need to solve the lack of
> >
> > committers and the lack of contributors.
> >
> > On the lack of committers:
> >
> > 1. Ideally, there are some of the current Flink committers who have
> >
> > the bandwidth and can help with reviewing PRs and merging them.
> >
> > 2. If that's not an option, it could be a consideration that current
> >
> > committers only approve and review PRs, that are approved by those who
> are
> > willing to contribute to Statefun and if the CI passes
> >
> > On the lack of contributors:
> >
> > 3. Next to having this discussion on the Dev and User mailing list, we
> >
> > can also create a blog with a call for new contributors on the Flink
> > project website, send out some tweets on the Flink / Statefun twitter
> > accounts, post messages on Slack etc. In that message, we would inform
> how
> > those that are interested in contributing can start and where they could
> > reach out for more information.
> >
> > There's also option 4. where a group of interested people would split
> >
> > Statefun from the Flink project and make it a separate top level project
> > under the Apache Flink umbrella (similar as recently has happened with
> > Flink Table Store, which has become Apache Paimon).
> >
> > If we see no improvements in the coming period, we should consider
> >
> > sunsetting Statefun and communicate that clearly to the users.
> >
> > I'm looking forward to your thoughts.
> >
> > Best regards,
> >
> > Martijn
> >
> > [1] https://nightlies.apache.org/flink/flink-statefun-docs-master/ <
> >
> > https://nightlies.apache.org/flink/flink-statefun-docs-master/>
> >
> > [2] https://github.com/apache/flink-statefun <
> >
> > https://github.com/apache/flink-statefun>
> >
> >
> >
> >
> >
>

Re: [DISCUSS] Status of Statefun Project

[Martijn Visser <ma...@apache.org>]

Hi all,

Apologies for the late reply.

I'm willing to help out with merging requests in Statefun to keep them
compatible with new Flink releases and create new releases. I do think that
validation of the functionality of these releases depends a lot on those
who do these compatibility updates, with PMC members helping out with the
formal process.

> Why can't the Apache Software Foundation allow community members to bring
it up to date?

There's nothing preventing anyone from reviewing any of the current PRs or
opening new ones. However, none of them are approved [1], so there's also
nothing to merge.

> I believe that there are people and companies on this mailing list
interested in supporting Apache Flink Stateful Functions.

If so, then now is the time to show.

Would there be a preference to create a release with Galen's merged
compatibility update to Flink 1.15.2, or do we want to skip that and go
straight to a newer version?

Best regards,

Martijn

[1]
https://github.com/apache/flink-statefun/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved

On Tue, Jun 6, 2023 at 3:55 PM Marco Villalobos <mv...@kineteque.com>
wrote:

> Why can't the Apache Software Foundation allow community members to bring
> it up to date?
>
> What's the process for that?
>
> I believe that there are people and companies on this mailing list
> interested in supporting Apache Flink Stateful Functions.
>
> You already had two people on this thread express interest.
>
> At the very least, we could keep the library versions up to date.
>
> There are only a small list of new features that might be worthwhile:
>
> 1. event time processing
> 2. state rest api
>
>
> On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <ch...@apache.org> wrote:
>
> If you were to fork it *and want to redistribute it* then the short
> version is that
>
>    1. you have to adhere to the Apache licensing requirements
>    2. you have to make it clear that your fork does not belong to the
>    Apache Flink project. (Trademarks and all that)
>
> Neither should be significant hurdles (there should also be plenty of
> online resources regarding 1), and if you do this then you can freely share
> your fork with others.
>
> I've also pinged Martijn to take a look at this thread.
> To my knowledge the project hasn't decided anything yet.
>
> On 27/05/2023 04:05, Galen Warren wrote:
>
> Ok, I get it. No interest.
>
> If this project is being abandoned, I guess I'll work with my own fork. Is
> there anything I should consider here? Can I share it with other people who
> use this project?
>
> On Tue, May 16, 2023 at 10:50 AM Galen Warren <ga...@cvillewarrens.com> <ga...@cvillewarrens.com>
> wrote:
>
>
> Hi Martijn, since you opened this discussion thread, I'm curious what your
> thoughts are in light of the responses? Thanks.
>
> On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <ga...@cvillewarrens.com> <ga...@cvillewarrens.com>
> wrote:
>
>
> I use Apache Flink for stream processing, and StateFun as a hand-off
>
> point for the rest of the application.
> It serves well as a bridge between a Flink Streaming job and
> micro-services.
>
> This is essentially how I use it as well, and I would also be sad to see
> it sunsetted. It works well; I don't know that there is a lot of new
> development required, but if there are no new Statefun releases, then
> Statefun can only be used with older Flink versions.
>
> On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <mv...@kineteque.com> wrote:
>
>
> I am currently using Stateful Functions in my application.
>
> I use Apache Flink for stream processing, and StateFun as a hand-off
> point for the rest of the application.
> It serves well as a bridge between a Flink Streaming job and
> micro-services.
>
> I would be disappointed if StateFun was sunsetted.  Its a good idea.
>
> If there is anything I can do to help, as a contributor perhaps, please
> let me know.
>
>
> On Apr 3, 2023, at 2:02 AM, Martijn Visser <ma...@apache.org> <ma...@apache.org>
>
> wrote:
>
> Hi everyone,
>
> I want to open a discussion on the status of the Statefun Project [1]
>
> in Apache Flink. As you might have noticed, there hasn't been much
> development over the past months in the Statefun repository [2]. There is
> currently a lack of active contributors and committers who are able to help
> with the maintenance of the project.
>
> In order to improve the situation, we need to solve the lack of
>
> committers and the lack of contributors.
>
> On the lack of committers:
>
> 1. Ideally, there are some of the current Flink committers who have
>
> the bandwidth and can help with reviewing PRs and merging them.
>
> 2. If that's not an option, it could be a consideration that current
>
> committers only approve and review PRs, that are approved by those who are
> willing to contribute to Statefun and if the CI passes
>
> On the lack of contributors:
>
> 3. Next to having this discussion on the Dev and User mailing list, we
>
> can also create a blog with a call for new contributors on the Flink
> project website, send out some tweets on the Flink / Statefun twitter
> accounts, post messages on Slack etc. In that message, we would inform how
> those that are interested in contributing can start and where they could
> reach out for more information.
>
> There's also option 4. where a group of interested people would split
>
> Statefun from the Flink project and make it a separate top level project
> under the Apache Flink umbrella (similar as recently has happened with
> Flink Table Store, which has become Apache Paimon).
>
> If we see no improvements in the coming period, we should consider
>
> sunsetting Statefun and communicate that clearly to the users.
>
> I'm looking forward to your thoughts.
>
> Best regards,
>
> Martijn
>
> [1] https://nightlies.apache.org/flink/flink-statefun-docs-master/ <
>
> https://nightlies.apache.org/flink/flink-statefun-docs-master/>
>
> [2] https://github.com/apache/flink-statefun <
>
> https://github.com/apache/flink-statefun>
>
>
>
>
>

Re: [DISCUSS] Status of Statefun Project

[Martijn Visser <ma...@apache.org>]

Hi all,

Apologies for the late reply.

I'm willing to help out with merging requests in Statefun to keep them
compatible with new Flink releases and create new releases. I do think that
validation of the functionality of these releases depends a lot on those
who do these compatibility updates, with PMC members helping out with the
formal process.

> Why can't the Apache Software Foundation allow community members to bring
it up to date?

There's nothing preventing anyone from reviewing any of the current PRs or
opening new ones. However, none of them are approved [1], so there's also
nothing to merge.

> I believe that there are people and companies on this mailing list
interested in supporting Apache Flink Stateful Functions.

If so, then now is the time to show.

Would there be a preference to create a release with Galen's merged
compatibility update to Flink 1.15.2, or do we want to skip that and go
straight to a newer version?

Best regards,

Martijn

[1]
https://github.com/apache/flink-statefun/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved

On Tue, Jun 6, 2023 at 3:55 PM Marco Villalobos <mv...@kineteque.com>
wrote:

> Why can't the Apache Software Foundation allow community members to bring
> it up to date?
>
> What's the process for that?
>
> I believe that there are people and companies on this mailing list
> interested in supporting Apache Flink Stateful Functions.
>
> You already had two people on this thread express interest.
>
> At the very least, we could keep the library versions up to date.
>
> There are only a small list of new features that might be worthwhile:
>
> 1. event time processing
> 2. state rest api
>
>
> On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <ch...@apache.org> wrote:
>
> If you were to fork it *and want to redistribute it* then the short
> version is that
>
>    1. you have to adhere to the Apache licensing requirements
>    2. you have to make it clear that your fork does not belong to the
>    Apache Flink project. (Trademarks and all that)
>
> Neither should be significant hurdles (there should also be plenty of
> online resources regarding 1), and if you do this then you can freely share
> your fork with others.
>
> I've also pinged Martijn to take a look at this thread.
> To my knowledge the project hasn't decided anything yet.
>
> On 27/05/2023 04:05, Galen Warren wrote:
>
> Ok, I get it. No interest.
>
> If this project is being abandoned, I guess I'll work with my own fork. Is
> there anything I should consider here? Can I share it with other people who
> use this project?
>
> On Tue, May 16, 2023 at 10:50 AM Galen Warren <ga...@cvillewarrens.com> <ga...@cvillewarrens.com>
> wrote:
>
>
> Hi Martijn, since you opened this discussion thread, I'm curious what your
> thoughts are in light of the responses? Thanks.
>
> On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <ga...@cvillewarrens.com> <ga...@cvillewarrens.com>
> wrote:
>
>
> I use Apache Flink for stream processing, and StateFun as a hand-off
>
> point for the rest of the application.
> It serves well as a bridge between a Flink Streaming job and
> micro-services.
>
> This is essentially how I use it as well, and I would also be sad to see
> it sunsetted. It works well; I don't know that there is a lot of new
> development required, but if there are no new Statefun releases, then
> Statefun can only be used with older Flink versions.
>
> On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <mv...@kineteque.com> wrote:
>
>
> I am currently using Stateful Functions in my application.
>
> I use Apache Flink for stream processing, and StateFun as a hand-off
> point for the rest of the application.
> It serves well as a bridge between a Flink Streaming job and
> micro-services.
>
> I would be disappointed if StateFun was sunsetted.  Its a good idea.
>
> If there is anything I can do to help, as a contributor perhaps, please
> let me know.
>
>
> On Apr 3, 2023, at 2:02 AM, Martijn Visser <ma...@apache.org> <ma...@apache.org>
>
> wrote:
>
> Hi everyone,
>
> I want to open a discussion on the status of the Statefun Project [1]
>
> in Apache Flink. As you might have noticed, there hasn't been much
> development over the past months in the Statefun repository [2]. There is
> currently a lack of active contributors and committers who are able to help
> with the maintenance of the project.
>
> In order to improve the situation, we need to solve the lack of
>
> committers and the lack of contributors.
>
> On the lack of committers:
>
> 1. Ideally, there are some of the current Flink committers who have
>
> the bandwidth and can help with reviewing PRs and merging them.
>
> 2. If that's not an option, it could be a consideration that current
>
> committers only approve and review PRs, that are approved by those who are
> willing to contribute to Statefun and if the CI passes
>
> On the lack of contributors:
>
> 3. Next to having this discussion on the Dev and User mailing list, we
>
> can also create a blog with a call for new contributors on the Flink
> project website, send out some tweets on the Flink / Statefun twitter
> accounts, post messages on Slack etc. In that message, we would inform how
> those that are interested in contributing can start and where they could
> reach out for more information.
>
> There's also option 4. where a group of interested people would split
>
> Statefun from the Flink project and make it a separate top level project
> under the Apache Flink umbrella (similar as recently has happened with
> Flink Table Store, which has become Apache Paimon).
>
> If we see no improvements in the coming period, we should consider
>
> sunsetting Statefun and communicate that clearly to the users.
>
> I'm looking forward to your thoughts.
>
> Best regards,
>
> Martijn
>
> [1] https://nightlies.apache.org/flink/flink-statefun-docs-master/ <
>
> https://nightlies.apache.org/flink/flink-statefun-docs-master/>
>
> [2] https://github.com/apache/flink-statefun <
>
> https://github.com/apache/flink-statefun>
>
>
>
>
>

Re: [DISCUSS] Status of Statefun Project

[Marco Villalobos <mv...@kineteque.com>]

Why can't the Apache Software Foundation allow community members to bring it up to date?

What's the process for that?

I believe that there are people and companies on this mailing list interested in supporting Apache Flink Stateful Functions.

You already had two people on this thread express interest.

At the very least, we could keep the library versions up to date.

There are only a small list of new features that might be worthwhile:

1. event time processing
2. state rest api


> On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <ch...@apache.org> wrote:
> 
> If you were to fork it and want to redistribute it then the short version is that 
> you have to adhere to the Apache licensing requirements
> you have to make it clear that your fork does not belong to the Apache Flink project. (Trademarks and all that)
> Neither should be significant hurdles (there should also be plenty of online resources regarding 1), and if you do this then you can freely share your fork with others.
> 
> I've also pinged Martijn to take a look at this thread.
> To my knowledge the project hasn't decided anything yet.
> 
> On 27/05/2023 04:05, Galen Warren wrote:
>> Ok, I get it. No interest.
>> 
>> If this project is being abandoned, I guess I'll work with my own fork. Is
>> there anything I should consider here? Can I share it with other people who
>> use this project?
>> 
>> On Tue, May 16, 2023 at 10:50 AM Galen Warren <ga...@cvillewarrens.com> <ma...@cvillewarrens.com>
>> wrote:
>> 
>>> Hi Martijn, since you opened this discussion thread, I'm curious what your
>>> thoughts are in light of the responses? Thanks.
>>> 
>>> On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <ga...@cvillewarrens.com> <ma...@cvillewarrens.com>
>>> wrote:
>>> 
>>>> I use Apache Flink for stream processing, and StateFun as a hand-off
>>>>> point for the rest of the application.
>>>>> It serves well as a bridge between a Flink Streaming job and
>>>>> micro-services.
>>>> 
>>>> This is essentially how I use it as well, and I would also be sad to see
>>>> it sunsetted. It works well; I don't know that there is a lot of new
>>>> development required, but if there are no new Statefun releases, then
>>>> Statefun can only be used with older Flink versions.
>>>> 
>>>> On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
>>>> mvillalobos@kineteque.com <ma...@kineteque.com>> wrote:
>>>> 
>>>>> I am currently using Stateful Functions in my application.
>>>>> 
>>>>> I use Apache Flink for stream processing, and StateFun as a hand-off
>>>>> point for the rest of the application.
>>>>> It serves well as a bridge between a Flink Streaming job and
>>>>> micro-services.
>>>>> 
>>>>> I would be disappointed if StateFun was sunsetted.  Its a good idea.
>>>>> 
>>>>> If there is anything I can do to help, as a contributor perhaps, please
>>>>> let me know.
>>>>> 
>>>>>> On Apr 3, 2023, at 2:02 AM, Martijn Visser <ma...@apache.org> <ma...@apache.org>
>>>>> wrote:
>>>>>> Hi everyone,
>>>>>> 
>>>>>> I want to open a discussion on the status of the Statefun Project [1]
>>>>> in Apache Flink. As you might have noticed, there hasn't been much
>>>>> development over the past months in the Statefun repository [2]. There is
>>>>> currently a lack of active contributors and committers who are able to help
>>>>> with the maintenance of the project.
>>>>>> In order to improve the situation, we need to solve the lack of
>>>>> committers and the lack of contributors.
>>>>>> On the lack of committers:
>>>>>> 
>>>>>> 1. Ideally, there are some of the current Flink committers who have
>>>>> the bandwidth and can help with reviewing PRs and merging them.
>>>>>> 2. If that's not an option, it could be a consideration that current
>>>>> committers only approve and review PRs, that are approved by those who are
>>>>> willing to contribute to Statefun and if the CI passes
>>>>>> On the lack of contributors:
>>>>>> 
>>>>>> 3. Next to having this discussion on the Dev and User mailing list, we
>>>>> can also create a blog with a call for new contributors on the Flink
>>>>> project website, send out some tweets on the Flink / Statefun twitter
>>>>> accounts, post messages on Slack etc. In that message, we would inform how
>>>>> those that are interested in contributing can start and where they could
>>>>> reach out for more information.
>>>>>> There's also option 4. where a group of interested people would split
>>>>> Statefun from the Flink project and make it a separate top level project
>>>>> under the Apache Flink umbrella (similar as recently has happened with
>>>>> Flink Table Store, which has become Apache Paimon).
>>>>>> If we see no improvements in the coming period, we should consider
>>>>> sunsetting Statefun and communicate that clearly to the users.
>>>>>> I'm looking forward to your thoughts.
>>>>>> 
>>>>>> Best regards,
>>>>>> 
>>>>>> Martijn
>>>>>> 
>>>>>> [1] https://nightlies.apache.org/flink/flink-statefun-docs-master/ <https://nightlies.apache.org/flink/flink-statefun-docs-master/> <
>>>>> https://nightlies.apache.org/flink/flink-statefun-docs-master/ <https://nightlies.apache.org/flink/flink-statefun-docs-master/>>
>>>>>> [2] https://github.com/apache/flink-statefun <https://github.com/apache/flink-statefun> <
>>>>> https://github.com/apache/flink-statefun <https://github.com/apache/flink-statefun>>
>>>>> 
> 

Re: [DISCUSS] Status of Statefun Project

[Galen Warren via user <us...@flink.apache.org>]

Thanks for the information.

On Tue, Jun 6, 2023, 6:07 AM Chesnay Schepler <ch...@apache.org> wrote:

> If you were to fork it /and want to redistribute it/ then the short
> version is that
>
>  1. you have to adhere to the Apache licensing requirements
>  2. you have to make it clear that your fork does not belong to the
>     Apache Flink project. (Trademarks and all that)
>
> Neither should be significant hurdles (there should also be plenty of
> online resources regarding 1), and if you do this then you can freely
> share your fork with others.
>
> I've also pinged Martijn to take a look at this thread.
> To my knowledge the project hasn't decided anything yet.
>
> On 27/05/2023 04:05, Galen Warren wrote:
> > Ok, I get it. No interest.
> >
> > If this project is being abandoned, I guess I'll work with my own fork.
> Is
> > there anything I should consider here? Can I share it with other people
> who
> > use this project?
> >
> > On Tue, May 16, 2023 at 10:50 AM Galen Warren<ga...@cvillewarrens.com>
> > wrote:
> >
> >> Hi Martijn, since you opened this discussion thread, I'm curious what
> your
> >> thoughts are in light of the responses? Thanks.
> >>
> >> On Wed, Apr 19, 2023 at 1:21 PM Galen Warren<ga...@cvillewarrens.com>
> >> wrote:
> >>
> >>> I use Apache Flink for stream processing, and StateFun as a hand-off
> >>>> point for the rest of the application.
> >>>> It serves well as a bridge between a Flink Streaming job and
> >>>> micro-services.
> >>>
> >>> This is essentially how I use it as well, and I would also be sad to
> see
> >>> it sunsetted. It works well; I don't know that there is a lot of new
> >>> development required, but if there are no new Statefun releases, then
> >>> Statefun can only be used with older Flink versions.
> >>>
> >>> On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
> >>> mvillalobos@kineteque.com> wrote:
> >>>
> >>>> I am currently using Stateful Functions in my application.
> >>>>
> >>>> I use Apache Flink for stream processing, and StateFun as a hand-off
> >>>> point for the rest of the application.
> >>>> It serves well as a bridge between a Flink Streaming job and
> >>>> micro-services.
> >>>>
> >>>> I would be disappointed if StateFun was sunsetted.  Its a good idea.
> >>>>
> >>>> If there is anything I can do to help, as a contributor perhaps,
> please
> >>>> let me know.
> >>>>
> >>>>> On Apr 3, 2023, at 2:02 AM, Martijn Visser<ma...@apache.org>
> >>>> wrote:
> >>>>> Hi everyone,
> >>>>>
> >>>>> I want to open a discussion on the status of the Statefun Project [1]
> >>>> in Apache Flink. As you might have noticed, there hasn't been much
> >>>> development over the past months in the Statefun repository [2].
> There is
> >>>> currently a lack of active contributors and committers who are able
> to help
> >>>> with the maintenance of the project.
> >>>>> In order to improve the situation, we need to solve the lack of
> >>>> committers and the lack of contributors.
> >>>>> On the lack of committers:
> >>>>>
> >>>>> 1. Ideally, there are some of the current Flink committers who have
> >>>> the bandwidth and can help with reviewing PRs and merging them.
> >>>>> 2. If that's not an option, it could be a consideration that current
> >>>> committers only approve and review PRs, that are approved by those
> who are
> >>>> willing to contribute to Statefun and if the CI passes
> >>>>> On the lack of contributors:
> >>>>>
> >>>>> 3. Next to having this discussion on the Dev and User mailing list,
> we
> >>>> can also create a blog with a call for new contributors on the Flink
> >>>> project website, send out some tweets on the Flink / Statefun twitter
> >>>> accounts, post messages on Slack etc. In that message, we would
> inform how
> >>>> those that are interested in contributing can start and where they
> could
> >>>> reach out for more information.
> >>>>> There's also option 4. where a group of interested people would split
> >>>> Statefun from the Flink project and make it a separate top level
> project
> >>>> under the Apache Flink umbrella (similar as recently has happened with
> >>>> Flink Table Store, which has become Apache Paimon).
> >>>>> If we see no improvements in the coming period, we should consider
> >>>> sunsetting Statefun and communicate that clearly to the users.
> >>>>> I'm looking forward to your thoughts.
> >>>>>
> >>>>> Best regards,
> >>>>>
> >>>>> Martijn
> >>>>>
> >>>>> [1]https://nightlies.apache.org/flink/flink-statefun-docs-master/  <
> >>>> https://nightlies.apache.org/flink/flink-statefun-docs-master/>
> >>>>> [2]https://github.com/apache/flink-statefun  <
> >>>> https://github.com/apache/flink-statefun>
> >>>>
>

Re: [DISCUSS] Status of Statefun Project

[Marco Villalobos <mv...@kineteque.com>]

Why can't the Apache Software Foundation allow community members to bring it up to date?

What's the process for that?

I believe that there are people and companies on this mailing list interested in supporting Apache Flink Stateful Functions.

You already had two people on this thread express interest.

At the very least, we could keep the library versions up to date.

There are only a small list of new features that might be worthwhile:

1. event time processing
2. state rest api


> On Jun 6, 2023, at 3:06 AM, Chesnay Schepler <ch...@apache.org> wrote:
> 
> If you were to fork it and want to redistribute it then the short version is that 
> you have to adhere to the Apache licensing requirements
> you have to make it clear that your fork does not belong to the Apache Flink project. (Trademarks and all that)
> Neither should be significant hurdles (there should also be plenty of online resources regarding 1), and if you do this then you can freely share your fork with others.
> 
> I've also pinged Martijn to take a look at this thread.
> To my knowledge the project hasn't decided anything yet.
> 
> On 27/05/2023 04:05, Galen Warren wrote:
>> Ok, I get it. No interest.
>> 
>> If this project is being abandoned, I guess I'll work with my own fork. Is
>> there anything I should consider here? Can I share it with other people who
>> use this project?
>> 
>> On Tue, May 16, 2023 at 10:50 AM Galen Warren <ga...@cvillewarrens.com> <ma...@cvillewarrens.com>
>> wrote:
>> 
>>> Hi Martijn, since you opened this discussion thread, I'm curious what your
>>> thoughts are in light of the responses? Thanks.
>>> 
>>> On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <ga...@cvillewarrens.com> <ma...@cvillewarrens.com>
>>> wrote:
>>> 
>>>> I use Apache Flink for stream processing, and StateFun as a hand-off
>>>>> point for the rest of the application.
>>>>> It serves well as a bridge between a Flink Streaming job and
>>>>> micro-services.
>>>> 
>>>> This is essentially how I use it as well, and I would also be sad to see
>>>> it sunsetted. It works well; I don't know that there is a lot of new
>>>> development required, but if there are no new Statefun releases, then
>>>> Statefun can only be used with older Flink versions.
>>>> 
>>>> On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
>>>> mvillalobos@kineteque.com <ma...@kineteque.com>> wrote:
>>>> 
>>>>> I am currently using Stateful Functions in my application.
>>>>> 
>>>>> I use Apache Flink for stream processing, and StateFun as a hand-off
>>>>> point for the rest of the application.
>>>>> It serves well as a bridge between a Flink Streaming job and
>>>>> micro-services.
>>>>> 
>>>>> I would be disappointed if StateFun was sunsetted.  Its a good idea.
>>>>> 
>>>>> If there is anything I can do to help, as a contributor perhaps, please
>>>>> let me know.
>>>>> 
>>>>>> On Apr 3, 2023, at 2:02 AM, Martijn Visser <ma...@apache.org> <ma...@apache.org>
>>>>> wrote:
>>>>>> Hi everyone,
>>>>>> 
>>>>>> I want to open a discussion on the status of the Statefun Project [1]
>>>>> in Apache Flink. As you might have noticed, there hasn't been much
>>>>> development over the past months in the Statefun repository [2]. There is
>>>>> currently a lack of active contributors and committers who are able to help
>>>>> with the maintenance of the project.
>>>>>> In order to improve the situation, we need to solve the lack of
>>>>> committers and the lack of contributors.
>>>>>> On the lack of committers:
>>>>>> 
>>>>>> 1. Ideally, there are some of the current Flink committers who have
>>>>> the bandwidth and can help with reviewing PRs and merging them.
>>>>>> 2. If that's not an option, it could be a consideration that current
>>>>> committers only approve and review PRs, that are approved by those who are
>>>>> willing to contribute to Statefun and if the CI passes
>>>>>> On the lack of contributors:
>>>>>> 
>>>>>> 3. Next to having this discussion on the Dev and User mailing list, we
>>>>> can also create a blog with a call for new contributors on the Flink
>>>>> project website, send out some tweets on the Flink / Statefun twitter
>>>>> accounts, post messages on Slack etc. In that message, we would inform how
>>>>> those that are interested in contributing can start and where they could
>>>>> reach out for more information.
>>>>>> There's also option 4. where a group of interested people would split
>>>>> Statefun from the Flink project and make it a separate top level project
>>>>> under the Apache Flink umbrella (similar as recently has happened with
>>>>> Flink Table Store, which has become Apache Paimon).
>>>>>> If we see no improvements in the coming period, we should consider
>>>>> sunsetting Statefun and communicate that clearly to the users.
>>>>>> I'm looking forward to your thoughts.
>>>>>> 
>>>>>> Best regards,
>>>>>> 
>>>>>> Martijn
>>>>>> 
>>>>>> [1] https://nightlies.apache.org/flink/flink-statefun-docs-master/ <https://nightlies.apache.org/flink/flink-statefun-docs-master/> <
>>>>> https://nightlies.apache.org/flink/flink-statefun-docs-master/ <https://nightlies.apache.org/flink/flink-statefun-docs-master/>>
>>>>>> [2] https://github.com/apache/flink-statefun <https://github.com/apache/flink-statefun> <
>>>>> https://github.com/apache/flink-statefun <https://github.com/apache/flink-statefun>>
>>>>> 
> 

Re: [DISCUSS] Status of Statefun Project

[Chesnay Schepler <ch...@apache.org>]

If you were to fork it /and want to redistribute it/ then the short 
version is that

 1. you have to adhere to the Apache licensing requirements
 2. you have to make it clear that your fork does not belong to the
    Apache Flink project. (Trademarks and all that)

Neither should be significant hurdles (there should also be plenty of 
online resources regarding 1), and if you do this then you can freely 
share your fork with others.

I've also pinged Martijn to take a look at this thread.
To my knowledge the project hasn't decided anything yet.

On 27/05/2023 04:05, Galen Warren wrote:
> Ok, I get it. No interest.
>
> If this project is being abandoned, I guess I'll work with my own fork. Is
> there anything I should consider here? Can I share it with other people who
> use this project?
>
> On Tue, May 16, 2023 at 10:50 AM Galen Warren<ga...@cvillewarrens.com>
> wrote:
>
>> Hi Martijn, since you opened this discussion thread, I'm curious what your
>> thoughts are in light of the responses? Thanks.
>>
>> On Wed, Apr 19, 2023 at 1:21 PM Galen Warren<ga...@cvillewarrens.com>
>> wrote:
>>
>>> I use Apache Flink for stream processing, and StateFun as a hand-off
>>>> point for the rest of the application.
>>>> It serves well as a bridge between a Flink Streaming job and
>>>> micro-services.
>>>
>>> This is essentially how I use it as well, and I would also be sad to see
>>> it sunsetted. It works well; I don't know that there is a lot of new
>>> development required, but if there are no new Statefun releases, then
>>> Statefun can only be used with older Flink versions.
>>>
>>> On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
>>> mvillalobos@kineteque.com> wrote:
>>>
>>>> I am currently using Stateful Functions in my application.
>>>>
>>>> I use Apache Flink for stream processing, and StateFun as a hand-off
>>>> point for the rest of the application.
>>>> It serves well as a bridge between a Flink Streaming job and
>>>> micro-services.
>>>>
>>>> I would be disappointed if StateFun was sunsetted.  Its a good idea.
>>>>
>>>> If there is anything I can do to help, as a contributor perhaps, please
>>>> let me know.
>>>>
>>>>> On Apr 3, 2023, at 2:02 AM, Martijn Visser<ma...@apache.org>
>>>> wrote:
>>>>> Hi everyone,
>>>>>
>>>>> I want to open a discussion on the status of the Statefun Project [1]
>>>> in Apache Flink. As you might have noticed, there hasn't been much
>>>> development over the past months in the Statefun repository [2]. There is
>>>> currently a lack of active contributors and committers who are able to help
>>>> with the maintenance of the project.
>>>>> In order to improve the situation, we need to solve the lack of
>>>> committers and the lack of contributors.
>>>>> On the lack of committers:
>>>>>
>>>>> 1. Ideally, there are some of the current Flink committers who have
>>>> the bandwidth and can help with reviewing PRs and merging them.
>>>>> 2. If that's not an option, it could be a consideration that current
>>>> committers only approve and review PRs, that are approved by those who are
>>>> willing to contribute to Statefun and if the CI passes
>>>>> On the lack of contributors:
>>>>>
>>>>> 3. Next to having this discussion on the Dev and User mailing list, we
>>>> can also create a blog with a call for new contributors on the Flink
>>>> project website, send out some tweets on the Flink / Statefun twitter
>>>> accounts, post messages on Slack etc. In that message, we would inform how
>>>> those that are interested in contributing can start and where they could
>>>> reach out for more information.
>>>>> There's also option 4. where a group of interested people would split
>>>> Statefun from the Flink project and make it a separate top level project
>>>> under the Apache Flink umbrella (similar as recently has happened with
>>>> Flink Table Store, which has become Apache Paimon).
>>>>> If we see no improvements in the coming period, we should consider
>>>> sunsetting Statefun and communicate that clearly to the users.
>>>>> I'm looking forward to your thoughts.
>>>>>
>>>>> Best regards,
>>>>>
>>>>> Martijn
>>>>>
>>>>> [1]https://nightlies.apache.org/flink/flink-statefun-docs-master/  <
>>>> https://nightlies.apache.org/flink/flink-statefun-docs-master/>
>>>>> [2]https://github.com/apache/flink-statefun  <
>>>> https://github.com/apache/flink-statefun>
>>>>

Re: [DISCUSS] Status of Statefun Project

[Galen Warren via user <us...@flink.apache.org>]

Getting to a resolution here would be great and much appreciated, yes.

On Sat, May 27, 2023 at 1:03 AM Salva Alcántara <sa...@gmail.com>
wrote:

> Hey Galen,
>
> I took a look at StateFun some time ago; not using it in production but I
> agree that it would be a pity to abandon it.
>
> As Martijn said, let's be clear on what lies ahead (what option is finally
> picked) and how to contribute (if possible) moving forward.
>
> Regards,
>
> Salva
>
> On Sat, May 27, 2023 at 4:05 AM Galen Warren via user <
> user@flink.apache.org> wrote:
>
>> Ok, I get it. No interest.
>>
>> If this project is being abandoned, I guess I'll work with my own fork.
>> Is there anything I should consider here? Can I share it with other people
>> who use this project?
>>
>> On Tue, May 16, 2023 at 10:50 AM Galen Warren <ga...@cvillewarrens.com>
>> wrote:
>>
>>> Hi Martijn, since you opened this discussion thread, I'm curious what
>>> your thoughts are in light of the responses? Thanks.
>>>
>>> On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <ga...@cvillewarrens.com>
>>> wrote:
>>>
>>>> I use Apache Flink for stream processing, and StateFun as a hand-off
>>>>> point for the rest of the application.
>>>>> It serves well as a bridge between a Flink Streaming job and
>>>>> micro-services.
>>>>
>>>>
>>>> This is essentially how I use it as well, and I would also be sad to
>>>> see it sunsetted. It works well; I don't know that there is a lot of new
>>>> development required, but if there are no new Statefun releases, then
>>>> Statefun can only be used with older Flink versions.
>>>>
>>>> On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
>>>> mvillalobos@kineteque.com> wrote:
>>>>
>>>>> I am currently using Stateful Functions in my application.
>>>>>
>>>>> I use Apache Flink for stream processing, and StateFun as a hand-off
>>>>> point for the rest of the application.
>>>>> It serves well as a bridge between a Flink Streaming job and
>>>>> micro-services.
>>>>>
>>>>> I would be disappointed if StateFun was sunsetted.  Its a good idea.
>>>>>
>>>>> If there is anything I can do to help, as a contributor perhaps,
>>>>> please let me know.
>>>>>
>>>>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <ma...@apache.org>
>>>>> wrote:
>>>>> >
>>>>> > Hi everyone,
>>>>> >
>>>>> > I want to open a discussion on the status of the Statefun Project
>>>>> [1] in Apache Flink. As you might have noticed, there hasn't been much
>>>>> development over the past months in the Statefun repository [2]. There is
>>>>> currently a lack of active contributors and committers who are able to help
>>>>> with the maintenance of the project.
>>>>> >
>>>>> > In order to improve the situation, we need to solve the lack of
>>>>> committers and the lack of contributors.
>>>>> >
>>>>> > On the lack of committers:
>>>>> >
>>>>> > 1. Ideally, there are some of the current Flink committers who have
>>>>> the bandwidth and can help with reviewing PRs and merging them.
>>>>> > 2. If that's not an option, it could be a consideration that current
>>>>> committers only approve and review PRs, that are approved by those who are
>>>>> willing to contribute to Statefun and if the CI passes
>>>>> >
>>>>> > On the lack of contributors:
>>>>> >
>>>>> > 3. Next to having this discussion on the Dev and User mailing list,
>>>>> we can also create a blog with a call for new contributors on the Flink
>>>>> project website, send out some tweets on the Flink / Statefun twitter
>>>>> accounts, post messages on Slack etc. In that message, we would inform how
>>>>> those that are interested in contributing can start and where they could
>>>>> reach out for more information.
>>>>> >
>>>>> > There's also option 4. where a group of interested people would
>>>>> split Statefun from the Flink project and make it a separate top level
>>>>> project under the Apache Flink umbrella (similar as recently has happened
>>>>> with Flink Table Store, which has become Apache Paimon).
>>>>> >
>>>>> > If we see no improvements in the coming period, we should consider
>>>>> sunsetting Statefun and communicate that clearly to the users.
>>>>> >
>>>>> > I'm looking forward to your thoughts.
>>>>> >
>>>>> > Best regards,
>>>>> >
>>>>> > Martijn
>>>>> >
>>>>> > [1] https://nightlies.apache.org/flink/flink-statefun-docs-master/ <
>>>>> https://nightlies.apache.org/flink/flink-statefun-docs-master/>
>>>>> > [2] https://github.com/apache/flink-statefun <
>>>>> https://github.com/apache/flink-statefun>
>>>>>
>>>>

Re: [DISCUSS] Status of Statefun Project

[Galen Warren <ga...@cvillewarrens.com.INVALID>]

Getting to a resolution here would be great and much appreciated, yes.

On Sat, May 27, 2023 at 1:03 AM Salva Alcántara <sa...@gmail.com>
wrote:

> Hey Galen,
>
> I took a look at StateFun some time ago; not using it in production but I
> agree that it would be a pity to abandon it.
>
> As Martijn said, let's be clear on what lies ahead (what option is finally
> picked) and how to contribute (if possible) moving forward.
>
> Regards,
>
> Salva
>
> On Sat, May 27, 2023 at 4:05 AM Galen Warren via user <
> user@flink.apache.org> wrote:
>
>> Ok, I get it. No interest.
>>
>> If this project is being abandoned, I guess I'll work with my own fork.
>> Is there anything I should consider here? Can I share it with other people
>> who use this project?
>>
>> On Tue, May 16, 2023 at 10:50 AM Galen Warren <ga...@cvillewarrens.com>
>> wrote:
>>
>>> Hi Martijn, since you opened this discussion thread, I'm curious what
>>> your thoughts are in light of the responses? Thanks.
>>>
>>> On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <ga...@cvillewarrens.com>
>>> wrote:
>>>
>>>> I use Apache Flink for stream processing, and StateFun as a hand-off
>>>>> point for the rest of the application.
>>>>> It serves well as a bridge between a Flink Streaming job and
>>>>> micro-services.
>>>>
>>>>
>>>> This is essentially how I use it as well, and I would also be sad to
>>>> see it sunsetted. It works well; I don't know that there is a lot of new
>>>> development required, but if there are no new Statefun releases, then
>>>> Statefun can only be used with older Flink versions.
>>>>
>>>> On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
>>>> mvillalobos@kineteque.com> wrote:
>>>>
>>>>> I am currently using Stateful Functions in my application.
>>>>>
>>>>> I use Apache Flink for stream processing, and StateFun as a hand-off
>>>>> point for the rest of the application.
>>>>> It serves well as a bridge between a Flink Streaming job and
>>>>> micro-services.
>>>>>
>>>>> I would be disappointed if StateFun was sunsetted.  Its a good idea.
>>>>>
>>>>> If there is anything I can do to help, as a contributor perhaps,
>>>>> please let me know.
>>>>>
>>>>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <ma...@apache.org>
>>>>> wrote:
>>>>> >
>>>>> > Hi everyone,
>>>>> >
>>>>> > I want to open a discussion on the status of the Statefun Project
>>>>> [1] in Apache Flink. As you might have noticed, there hasn't been much
>>>>> development over the past months in the Statefun repository [2]. There is
>>>>> currently a lack of active contributors and committers who are able to help
>>>>> with the maintenance of the project.
>>>>> >
>>>>> > In order to improve the situation, we need to solve the lack of
>>>>> committers and the lack of contributors.
>>>>> >
>>>>> > On the lack of committers:
>>>>> >
>>>>> > 1. Ideally, there are some of the current Flink committers who have
>>>>> the bandwidth and can help with reviewing PRs and merging them.
>>>>> > 2. If that's not an option, it could be a consideration that current
>>>>> committers only approve and review PRs, that are approved by those who are
>>>>> willing to contribute to Statefun and if the CI passes
>>>>> >
>>>>> > On the lack of contributors:
>>>>> >
>>>>> > 3. Next to having this discussion on the Dev and User mailing list,
>>>>> we can also create a blog with a call for new contributors on the Flink
>>>>> project website, send out some tweets on the Flink / Statefun twitter
>>>>> accounts, post messages on Slack etc. In that message, we would inform how
>>>>> those that are interested in contributing can start and where they could
>>>>> reach out for more information.
>>>>> >
>>>>> > There's also option 4. where a group of interested people would
>>>>> split Statefun from the Flink project and make it a separate top level
>>>>> project under the Apache Flink umbrella (similar as recently has happened
>>>>> with Flink Table Store, which has become Apache Paimon).
>>>>> >
>>>>> > If we see no improvements in the coming period, we should consider
>>>>> sunsetting Statefun and communicate that clearly to the users.
>>>>> >
>>>>> > I'm looking forward to your thoughts.
>>>>> >
>>>>> > Best regards,
>>>>> >
>>>>> > Martijn
>>>>> >
>>>>> > [1] https://nightlies.apache.org/flink/flink-statefun-docs-master/ <
>>>>> https://nightlies.apache.org/flink/flink-statefun-docs-master/>
>>>>> > [2] https://github.com/apache/flink-statefun <
>>>>> https://github.com/apache/flink-statefun>
>>>>>
>>>>

Re: [DISCUSS] Status of Statefun Project

[Salva Alcántara <sa...@gmail.com>]

Hey Galen,

I took a look at StateFun some time ago; not using it in production but I
agree that it would be a pity to abandon it.

As Martijn said, let's be clear on what lies ahead (what option is finally
picked) and how to contribute (if possible) moving forward.

Regards,

Salva

On Sat, May 27, 2023 at 4:05 AM Galen Warren via user <us...@flink.apache.org>
wrote:

> Ok, I get it. No interest.
>
> If this project is being abandoned, I guess I'll work with my own fork. Is
> there anything I should consider here? Can I share it with other people who
> use this project?
>
> On Tue, May 16, 2023 at 10:50 AM Galen Warren <ga...@cvillewarrens.com>
> wrote:
>
>> Hi Martijn, since you opened this discussion thread, I'm curious what
>> your thoughts are in light of the responses? Thanks.
>>
>> On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <ga...@cvillewarrens.com>
>> wrote:
>>
>>> I use Apache Flink for stream processing, and StateFun as a hand-off
>>>> point for the rest of the application.
>>>> It serves well as a bridge between a Flink Streaming job and
>>>> micro-services.
>>>
>>>
>>> This is essentially how I use it as well, and I would also be sad to see
>>> it sunsetted. It works well; I don't know that there is a lot of new
>>> development required, but if there are no new Statefun releases, then
>>> Statefun can only be used with older Flink versions.
>>>
>>> On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
>>> mvillalobos@kineteque.com> wrote:
>>>
>>>> I am currently using Stateful Functions in my application.
>>>>
>>>> I use Apache Flink for stream processing, and StateFun as a hand-off
>>>> point for the rest of the application.
>>>> It serves well as a bridge between a Flink Streaming job and
>>>> micro-services.
>>>>
>>>> I would be disappointed if StateFun was sunsetted.  Its a good idea.
>>>>
>>>> If there is anything I can do to help, as a contributor perhaps, please
>>>> let me know.
>>>>
>>>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <ma...@apache.org>
>>>> wrote:
>>>> >
>>>> > Hi everyone,
>>>> >
>>>> > I want to open a discussion on the status of the Statefun Project [1]
>>>> in Apache Flink. As you might have noticed, there hasn't been much
>>>> development over the past months in the Statefun repository [2]. There is
>>>> currently a lack of active contributors and committers who are able to help
>>>> with the maintenance of the project.
>>>> >
>>>> > In order to improve the situation, we need to solve the lack of
>>>> committers and the lack of contributors.
>>>> >
>>>> > On the lack of committers:
>>>> >
>>>> > 1. Ideally, there are some of the current Flink committers who have
>>>> the bandwidth and can help with reviewing PRs and merging them.
>>>> > 2. If that's not an option, it could be a consideration that current
>>>> committers only approve and review PRs, that are approved by those who are
>>>> willing to contribute to Statefun and if the CI passes
>>>> >
>>>> > On the lack of contributors:
>>>> >
>>>> > 3. Next to having this discussion on the Dev and User mailing list,
>>>> we can also create a blog with a call for new contributors on the Flink
>>>> project website, send out some tweets on the Flink / Statefun twitter
>>>> accounts, post messages on Slack etc. In that message, we would inform how
>>>> those that are interested in contributing can start and where they could
>>>> reach out for more information.
>>>> >
>>>> > There's also option 4. where a group of interested people would split
>>>> Statefun from the Flink project and make it a separate top level project
>>>> under the Apache Flink umbrella (similar as recently has happened with
>>>> Flink Table Store, which has become Apache Paimon).
>>>> >
>>>> > If we see no improvements in the coming period, we should consider
>>>> sunsetting Statefun and communicate that clearly to the users.
>>>> >
>>>> > I'm looking forward to your thoughts.
>>>> >
>>>> > Best regards,
>>>> >
>>>> > Martijn
>>>> >
>>>> > [1] https://nightlies.apache.org/flink/flink-statefun-docs-master/ <
>>>> https://nightlies.apache.org/flink/flink-statefun-docs-master/>
>>>> > [2] https://github.com/apache/flink-statefun <
>>>> https://github.com/apache/flink-statefun>
>>>>
>>>

Re: [DISCUSS] Status of Statefun Project

[Chesnay Schepler <ch...@apache.org>]

If you were to fork it /and want to redistribute it/ then the short 
version is that

 1. you have to adhere to the Apache licensing requirements
 2. you have to make it clear that your fork does not belong to the
    Apache Flink project. (Trademarks and all that)

Neither should be significant hurdles (there should also be plenty of 
online resources regarding 1), and if you do this then you can freely 
share your fork with others.

I've also pinged Martijn to take a look at this thread.
To my knowledge the project hasn't decided anything yet.

On 27/05/2023 04:05, Galen Warren wrote:
> Ok, I get it. No interest.
>
> If this project is being abandoned, I guess I'll work with my own fork. Is
> there anything I should consider here? Can I share it with other people who
> use this project?
>
> On Tue, May 16, 2023 at 10:50 AM Galen Warren<ga...@cvillewarrens.com>
> wrote:
>
>> Hi Martijn, since you opened this discussion thread, I'm curious what your
>> thoughts are in light of the responses? Thanks.
>>
>> On Wed, Apr 19, 2023 at 1:21 PM Galen Warren<ga...@cvillewarrens.com>
>> wrote:
>>
>>> I use Apache Flink for stream processing, and StateFun as a hand-off
>>>> point for the rest of the application.
>>>> It serves well as a bridge between a Flink Streaming job and
>>>> micro-services.
>>>
>>> This is essentially how I use it as well, and I would also be sad to see
>>> it sunsetted. It works well; I don't know that there is a lot of new
>>> development required, but if there are no new Statefun releases, then
>>> Statefun can only be used with older Flink versions.
>>>
>>> On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
>>> mvillalobos@kineteque.com> wrote:
>>>
>>>> I am currently using Stateful Functions in my application.
>>>>
>>>> I use Apache Flink for stream processing, and StateFun as a hand-off
>>>> point for the rest of the application.
>>>> It serves well as a bridge between a Flink Streaming job and
>>>> micro-services.
>>>>
>>>> I would be disappointed if StateFun was sunsetted.  Its a good idea.
>>>>
>>>> If there is anything I can do to help, as a contributor perhaps, please
>>>> let me know.
>>>>
>>>>> On Apr 3, 2023, at 2:02 AM, Martijn Visser<ma...@apache.org>
>>>> wrote:
>>>>> Hi everyone,
>>>>>
>>>>> I want to open a discussion on the status of the Statefun Project [1]
>>>> in Apache Flink. As you might have noticed, there hasn't been much
>>>> development over the past months in the Statefun repository [2]. There is
>>>> currently a lack of active contributors and committers who are able to help
>>>> with the maintenance of the project.
>>>>> In order to improve the situation, we need to solve the lack of
>>>> committers and the lack of contributors.
>>>>> On the lack of committers:
>>>>>
>>>>> 1. Ideally, there are some of the current Flink committers who have
>>>> the bandwidth and can help with reviewing PRs and merging them.
>>>>> 2. If that's not an option, it could be a consideration that current
>>>> committers only approve and review PRs, that are approved by those who are
>>>> willing to contribute to Statefun and if the CI passes
>>>>> On the lack of contributors:
>>>>>
>>>>> 3. Next to having this discussion on the Dev and User mailing list, we
>>>> can also create a blog with a call for new contributors on the Flink
>>>> project website, send out some tweets on the Flink / Statefun twitter
>>>> accounts, post messages on Slack etc. In that message, we would inform how
>>>> those that are interested in contributing can start and where they could
>>>> reach out for more information.
>>>>> There's also option 4. where a group of interested people would split
>>>> Statefun from the Flink project and make it a separate top level project
>>>> under the Apache Flink umbrella (similar as recently has happened with
>>>> Flink Table Store, which has become Apache Paimon).
>>>>> If we see no improvements in the coming period, we should consider
>>>> sunsetting Statefun and communicate that clearly to the users.
>>>>> I'm looking forward to your thoughts.
>>>>>
>>>>> Best regards,
>>>>>
>>>>> Martijn
>>>>>
>>>>> [1]https://nightlies.apache.org/flink/flink-statefun-docs-master/  <
>>>> https://nightlies.apache.org/flink/flink-statefun-docs-master/>
>>>>> [2]https://github.com/apache/flink-statefun  <
>>>> https://github.com/apache/flink-statefun>
>>>>

Re: [DISCUSS] Status of Statefun Project

[Galen Warren <ga...@cvillewarrens.com.INVALID>]

Ok, I get it. No interest.

If this project is being abandoned, I guess I'll work with my own fork. Is
there anything I should consider here? Can I share it with other people who
use this project?

On Tue, May 16, 2023 at 10:50 AM Galen Warren <ga...@cvillewarrens.com>
wrote:

> Hi Martijn, since you opened this discussion thread, I'm curious what your
> thoughts are in light of the responses? Thanks.
>
> On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <ga...@cvillewarrens.com>
> wrote:
>
>> I use Apache Flink for stream processing, and StateFun as a hand-off
>>> point for the rest of the application.
>>> It serves well as a bridge between a Flink Streaming job and
>>> micro-services.
>>
>>
>> This is essentially how I use it as well, and I would also be sad to see
>> it sunsetted. It works well; I don't know that there is a lot of new
>> development required, but if there are no new Statefun releases, then
>> Statefun can only be used with older Flink versions.
>>
>> On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
>> mvillalobos@kineteque.com> wrote:
>>
>>> I am currently using Stateful Functions in my application.
>>>
>>> I use Apache Flink for stream processing, and StateFun as a hand-off
>>> point for the rest of the application.
>>> It serves well as a bridge between a Flink Streaming job and
>>> micro-services.
>>>
>>> I would be disappointed if StateFun was sunsetted.  Its a good idea.
>>>
>>> If there is anything I can do to help, as a contributor perhaps, please
>>> let me know.
>>>
>>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <ma...@apache.org>
>>> wrote:
>>> >
>>> > Hi everyone,
>>> >
>>> > I want to open a discussion on the status of the Statefun Project [1]
>>> in Apache Flink. As you might have noticed, there hasn't been much
>>> development over the past months in the Statefun repository [2]. There is
>>> currently a lack of active contributors and committers who are able to help
>>> with the maintenance of the project.
>>> >
>>> > In order to improve the situation, we need to solve the lack of
>>> committers and the lack of contributors.
>>> >
>>> > On the lack of committers:
>>> >
>>> > 1. Ideally, there are some of the current Flink committers who have
>>> the bandwidth and can help with reviewing PRs and merging them.
>>> > 2. If that's not an option, it could be a consideration that current
>>> committers only approve and review PRs, that are approved by those who are
>>> willing to contribute to Statefun and if the CI passes
>>> >
>>> > On the lack of contributors:
>>> >
>>> > 3. Next to having this discussion on the Dev and User mailing list, we
>>> can also create a blog with a call for new contributors on the Flink
>>> project website, send out some tweets on the Flink / Statefun twitter
>>> accounts, post messages on Slack etc. In that message, we would inform how
>>> those that are interested in contributing can start and where they could
>>> reach out for more information.
>>> >
>>> > There's also option 4. where a group of interested people would split
>>> Statefun from the Flink project and make it a separate top level project
>>> under the Apache Flink umbrella (similar as recently has happened with
>>> Flink Table Store, which has become Apache Paimon).
>>> >
>>> > If we see no improvements in the coming period, we should consider
>>> sunsetting Statefun and communicate that clearly to the users.
>>> >
>>> > I'm looking forward to your thoughts.
>>> >
>>> > Best regards,
>>> >
>>> > Martijn
>>> >
>>> > [1] https://nightlies.apache.org/flink/flink-statefun-docs-master/ <
>>> https://nightlies.apache.org/flink/flink-statefun-docs-master/>
>>> > [2] https://github.com/apache/flink-statefun <
>>> https://github.com/apache/flink-statefun>
>>>
>>

Re: [DISCUSS] Status of Statefun Project

[Galen Warren via user <us...@flink.apache.org>]

Ok, I get it. No interest.

If this project is being abandoned, I guess I'll work with my own fork. Is
there anything I should consider here? Can I share it with other people who
use this project?

On Tue, May 16, 2023 at 10:50 AM Galen Warren <ga...@cvillewarrens.com>
wrote:

> Hi Martijn, since you opened this discussion thread, I'm curious what your
> thoughts are in light of the responses? Thanks.
>
> On Wed, Apr 19, 2023 at 1:21 PM Galen Warren <ga...@cvillewarrens.com>
> wrote:
>
>> I use Apache Flink for stream processing, and StateFun as a hand-off
>>> point for the rest of the application.
>>> It serves well as a bridge between a Flink Streaming job and
>>> micro-services.
>>
>>
>> This is essentially how I use it as well, and I would also be sad to see
>> it sunsetted. It works well; I don't know that there is a lot of new
>> development required, but if there are no new Statefun releases, then
>> Statefun can only be used with older Flink versions.
>>
>> On Tue, Apr 18, 2023 at 10:04 PM Marco Villalobos <
>> mvillalobos@kineteque.com> wrote:
>>
>>> I am currently using Stateful Functions in my application.
>>>
>>> I use Apache Flink for stream processing, and StateFun as a hand-off
>>> point for the rest of the application.
>>> It serves well as a bridge between a Flink Streaming job and
>>> micro-services.
>>>
>>> I would be disappointed if StateFun was sunsetted.  Its a good idea.
>>>
>>> If there is anything I can do to help, as a contributor perhaps, please
>>> let me know.
>>>
>>> > On Apr 3, 2023, at 2:02 AM, Martijn Visser <ma...@apache.org>
>>> wrote:
>>> >
>>> > Hi everyone,
>>> >
>>> > I want to open a discussion on the status of the Statefun Project [1]
>>> in Apache Flink. As you might have noticed, there hasn't been much
>>> development over the past months in the Statefun repository [2]. There is
>>> currently a lack of active contributors and committers who are able to help
>>> with the maintenance of the project.
>>> >
>>> > In order to improve the situation, we need to solve the lack of
>>> committers and the lack of contributors.
>>> >
>>> > On the lack of committers:
>>> >
>>> > 1. Ideally, there are some of the current Flink committers who have
>>> the bandwidth and can help with reviewing PRs and merging them.
>>> > 2. If that's not an option, it could be a consideration that current
>>> committers only approve and review PRs, that are approved by those who are
>>> willing to contribute to Statefun and if the CI passes
>>> >
>>> > On the lack of contributors:
>>> >
>>> > 3. Next to having this discussion on the Dev and User mailing list, we
>>> can also create a blog with a call for new contributors on the Flink
>>> project website, send out some tweets on the Flink / Statefun twitter
>>> accounts, post messages on Slack etc. In that message, we would inform how
>>> those that are interested in contributing can start and where they could
>>> reach out for more information.
>>> >
>>> > There's also option 4. where a group of interested people would split
>>> Statefun from the Flink project and make it a separate top level project
>>> under the Apache Flink umbrella (similar as recently has happened with
>>> Flink Table Store, which has become Apache Paimon).
>>> >
>>> > If we see no improvements in the coming period, we should consider
>>> sunsetting Statefun and communicate that clearly to the users.
>>> >
>>> > I'm looking forward to your thoughts.
>>> >
>>> > Best regards,
>>> >
>>> > Martijn
>>> >
>>> > [1] https://nightlies.apache.org/flink/flink-statefun-docs-master/ <
>>> https://nightlies.apache.org/flink/flink-statefun-docs-master/>
>>> > [2] https://github.com/apache/flink-statefun <
>>> https://github.com/apache/flink-statefun>
>>>
>>