Re: [DISCUSS] StremaPark 2024 roadmap

[Yuepeng Pan <pa...@apache.org>]

Hi, devs.

Thanks a lot for the discussion  and thanks Huajie for driving it !

It would be the time to have a preliminary conclusion and list already:

Graduation related (high-priority)
●Graduate Apache StreamPark (Incubating) as a TLP 
●Document improvements

Stability & availability
●HA
●Apache Flink Metrics monitoring solutions
●Job  state consistence

Ecology support
●Support for Flink CDC 3.0
●Support Paimon
●catalog & datasource management
●Integrate Flink sql-gateway
●Refactor Flink on k8s using k8s-operator
●Support for PyFlink
●Support for Apache Spark (Development framework and platform) 

Please let's know what's your opinion for the mentioned above and why. 
BTW, If you are interested in any part or wish to contribute, please feel free to initiate a discussion in the mailing list.

Any suggestion and comment would be appreciated !

Best,
Yuepeng Pan

On 2024/03/25 02:37:09 Huajie Wang wrote:
> hi:
> 
> Welcome back here, The discussion is expected to end this week. Feel free
> to join the discussion if you have any new ideas.
> 
> 
> 
> Best,
> Huajie Wang
> 
> 
> 
> Huajie Wang <be...@apache.org> 于2024年3月4日周一 11:19写道：
> 
> > +1
> >
> > Metrics Monitoring are very important foundational capabilities. Apache
> > StreamPark(Incubating) needs to address this part of the capability better.
> > I strongly agree with this proposal.
> >
> > 2024 roadmap:
> >
> > 1. Support for PyFlink
> > 2. Support for Flink CDC 3.0
> > 3. Refactor Flink on k8s using k8s-operator
> >
> > 4. Integrate Flink sql-gateway
> >
> > 5. Support for Apache Spark(Development framework and platform) 6.
> > Document improvements
> >
> > 7. Graduate Apache StreamPark (Incubating) as a TLP
> >
> > 8. Apache Flink Metrics monitoring solutions
> >
> >
> > Best,
> > Huajie Wang
> >
> >
> >
> > shun SUN <su...@gmail.com> 于2024年3月4日周一 11:07写道：
> >
> >> Hi
> >>
> >> Thanks for the roadmap sharing, LGTM!
> >>
> >> Is it possible to add one point: provide monitoring  solutions for Flink
> >> jobs on yarn as well Kubernetes?
> >>
> >> I would like to take this part in coming months.
> >>
> >> Thanks and regards
> >> Sun Shun
> >>
> >> On Mon, 4 Mar 2024 at 10:58 AM, Huajie Wang <be...@apache.org> wrote:
> >>
> >> > I agree with your proposal:
> >> >
> >> > 1. Support for PyFlink
> >> > 2. Support for Flink CDC 3.0
> >> > 3. Refactor Flink on k8s using k8s-operator
> >> >
> >> > 4. Integrate Flink sql-gateway
> >> >
> >> > 5. Support for Apache Spark(Development framework and platform) 6.
> >> Document
> >> > improvements
> >> >
> >> > 7. Graduate Apache StreamPark (Incubating) as a TLP
> >> >
> >> > Welcome more discussions!
> >> >
> >> >
> >> >
> >> > Best,
> >> > Huajie Wang
> >> >
> >> >
> >> >
> >> > Cancai Cai <ca...@apache.org> 于2024年3月2日周六 19:34写道：
> >> >
> >> > > This looks good, +1.
> >> > > A small suggestion, should document improvements also be put into the
> >> > 2024
> >> > > roadmap?
> >> > >
> >> > > Huajie Wang <be...@apache.org> 于2024年3月2日周六 18:53写道：
> >> > >
> >> > > > Hello everyone:
> >> > > >
> >> > > > The support capability of  Apache StreamPark for Apache Flink has
> >> been
> >> > > > increasingly improved and stable., with more and more enterprise
> >> users
> >> > > > using it. This is the effort of all developers, and we are delighted
> >> > > > to see this. Now I would like to initiate a discussion on the
> >> > > > StreamPark 2024 roadmap, welcoming everyone to participate and share
> >> > > > their opinions.
> >> > > >
> >> > > > Here are some new features I have in mind for 2024:
> >> > > >
> >> > > > 1. Support for PyFlink
> >> > > > 2. Support for Flink CDC 3.0
> >> > > > 3. Refactor Flink on k8s using k8s-operator
> >> > > >
> >> > > > 4. Integrate Flink sql-gateway
> >> > > >
> >> > > > 5. Support for Apache Spark(Development framework and platform)
> >> > > >
> >> > > > Looking forward to your opinion. Let's make Apache StramPark even
> >> > better
> >> > > in
> >> > > > 2024! 🚀
> >> > > >
> >> > > >
> >> > > > Best,
> >> > > > Huajie Wang
> >> > > >
> >> > >
> >> >
> >>
> >
> 

Re:Re: [DISCUSS] StremaPark 2024 roadmap

[欧阳武林 <ou...@163.com>]

Hi, jianjun Xu 
  We can start by opening a design document.

















At 2024-04-12 16:17:01, "Jianjun Xu" <xu...@gmail.com> wrote:
>Hi, devs.
>
>> I am interested in catalogs and data source management modules. Currently,
>> we are investigating the functions of metadata related modules. The
>> construction of metadata related functions is expected to start next week,
>> and we hope to participate in the joint construction with the community then
>>
> Best,
>Jianjun Xu

Re: [DISCUSS] StremaPark 2024 roadmap

[Jianjun Xu <xu...@gmail.com>]

Hi, devs.

> I am interested in catalogs and data source management modules. Currently,
> we are investigating the functions of metadata related modules. The
> construction of metadata related functions is expected to start next week,
> and we hope to participate in the joint construction with the community then
>
 Best,
Jianjun Xu