You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flink.apache.org by bo zhaobo <bz...@gmail.com> on 2022/08/23 11:28:17 UTC

[DISCUSS] ARM support for Flink

Hi Flinkers,

In 2019, we raised an discussion in Flink about "ARM support for Flink"[1].
And we got so many helps and supports from Flink community about
introducing a ARM CI system into Flink community, which is named
"OpenLab"[2], and finally we create a full stack regression Flink tests on
OpenLab ARM resources, then post a email to Flink Maillist about test
result every day. We've been doing that for almost 2 years.

But now, we are so apologized that OpenLab had been reached its EOL. We had
to close it last month. So for sustaining the existing ARM CI still work on
Flink Community and helping contributors to verify their code on ARM. We
decide that we *will donate some ARM resources(Virtual Machines) *into
Flink community to make this happen.

And considering the existing Flink CICD had been moved to Azure Pipeline,
and doesn't use github action.

Now what we can provide are *ONLY *ARM resources(Virtual Machines) from us,
so we think *Flink community is the right role to decide how to use them.*
We only give several suggestions here:
1. Github action self-hosted Machines(Integrated with our ARM resources)
2. Azure Pipeline self-hosted Machines(Integrated with our ARM resources)
3. Any ideas from Flinkers?

If community accepts our ARM resources and want to integrate with existing
CICD in any way, please feel free to ping me about the quota(CPU nums,
Memory&Disk size and so on) of VM we need to donate.

Thank you very much.

BR

Bo Zhao

[1] https://www.mail-archive.com/dev@flink.apache.org/msg27054.html
[2] https://openlabtesting.org/

Re: [DISCUSS] ARM support for Flink

Posted by bo zhaobo <bz...@gmail.com>.

Hi,

Thanks for feeback and descirption, Yang Wang.

I saw multiple Flink associated docker images on dockerhub and released
with X86 & ARM64. So we think that might be useful for community to setup
ARM CI relied on existing CICD system.

We are happy that we could help on that via donating ARM resources. Also,
if Flink INFRA team could involve in this discussion, I think we can setup
the work. ;-)

Thanks again for info and advice.

BR

Bo Zhao

Yang Wang <da...@gmail.com> 于2022年8月26日周五 19:13写道：

> Thanks Bo for starting this discussion.
>
> I think it is really useful to have the CI for ARM platform. But I am not
> sure what is the current situation. TBH, I have not build an ARM image for
> Flink.
>
> Given that we have not finished the migration from azure pipelines to
> github action, I believe we still need to setup the self-hosted ARM
> machines for azure pipeline.
>
> Best,
> Yang
>
> bo zhaobo <bz...@gmail.com> 于2022年8月25日周四 09:37写道：
>
> > Hi Flink,
> > Any idea about this? ;-)
> >
> > bo zhaobo <bz...@gmail.com> 于2022年8月23日周二 19:28写道：
> >
> > > Hi Flinkers,
> > >
> > > In 2019, we raised an discussion in Flink about "ARM support for
> > > Flink"[1]. And we got so many helps and supports from Flink community
> > about
> > > introducing a ARM CI system into Flink community, which is named
> > > "OpenLab"[2], and finally we create a full stack regression Flink tests
> > on
> > > OpenLab ARM resources, then post a email to Flink Maillist about test
> > > result every day. We've been doing that for almost 2 years.
> > >
> > > But now, we are so apologized that OpenLab had been reached its EOL. We
> > > had to close it last month. So for sustaining the existing ARM CI still
> > > work on Flink Community and helping contributors to verify their code
> on
> > > ARM. We decide that we *will donate some ARM resources(Virtual
> Machines)
> > *into
> > > Flink community to make this happen.
> > >
> > > And considering the existing Flink CICD had been moved to Azure
> Pipeline,
> > > and doesn't use github action.
> > >
> > > Now what we can provide are *ONLY *ARM resources(Virtual Machines) from
> > > us, so we think *Flink community is the right role to decide how to use
> > > them.*
> > > We only give several suggestions here:
> > > 1. Github action self-hosted Machines(Integrated with our ARM
> resources)
> > > 2. Azure Pipeline self-hosted Machines(Integrated with our ARM
> resources)
> > > 3. Any ideas from Flinkers?
> > >
> > > If community accepts our ARM resources and want to integrate with
> > existing
> > > CICD in any way, please feel free to ping me about the quota(CPU nums,
> > > Memory&Disk size and so on) of VM we need to donate.
> > >
> > > Thank you very much.
> > >
> > > BR
> > >
> > > Bo Zhao
> > >
> > > [1] https://www.mail-archive.com/dev@flink.apache.org/msg27054.html
> > > [2] https://openlabtesting.org/
> > >
> >
>

Re: [DISCUSS] ARM support for Flink

Posted by Yang Wang <da...@gmail.com>.

Thanks Bo for starting this discussion.

I think it is really useful to have the CI for ARM platform. But I am not
sure what is the current situation. TBH, I have not build an ARM image for
Flink.

Given that we have not finished the migration from azure pipelines to
github action, I believe we still need to setup the self-hosted ARM
machines for azure pipeline.

Best,
Yang

bo zhaobo <bz...@gmail.com> 于2022年8月25日周四 09:37写道：

> Hi Flink,
> Any idea about this? ;-)
>
> bo zhaobo <bz...@gmail.com> 于2022年8月23日周二 19:28写道：
>
> > Hi Flinkers,
> >
> > In 2019, we raised an discussion in Flink about "ARM support for
> > Flink"[1]. And we got so many helps and supports from Flink community
> about
> > introducing a ARM CI system into Flink community, which is named
> > "OpenLab"[2], and finally we create a full stack regression Flink tests
> on
> > OpenLab ARM resources, then post a email to Flink Maillist about test
> > result every day. We've been doing that for almost 2 years.
> >
> > But now, we are so apologized that OpenLab had been reached its EOL. We
> > had to close it last month. So for sustaining the existing ARM CI still
> > work on Flink Community and helping contributors to verify their code on
> > ARM. We decide that we *will donate some ARM resources(Virtual Machines)
> *into
> > Flink community to make this happen.
> >
> > And considering the existing Flink CICD had been moved to Azure Pipeline,
> > and doesn't use github action.
> >
> > Now what we can provide are *ONLY *ARM resources(Virtual Machines) from
> > us, so we think *Flink community is the right role to decide how to use
> > them.*
> > We only give several suggestions here:
> > 1. Github action self-hosted Machines(Integrated with our ARM resources)
> > 2. Azure Pipeline self-hosted Machines(Integrated with our ARM resources)
> > 3. Any ideas from Flinkers?
> >
> > If community accepts our ARM resources and want to integrate with
> existing
> > CICD in any way, please feel free to ping me about the quota(CPU nums,
> > Memory&Disk size and so on) of VM we need to donate.
> >
> > Thank you very much.
> >
> > BR
> >
> > Bo Zhao
> >
> > [1] https://www.mail-archive.com/dev@flink.apache.org/msg27054.html
> > [2] https://openlabtesting.org/
> >
>

Re: [DISCUSS] ARM support for Flink

Posted by bo zhaobo <bz...@gmail.com>.

Hi Flink,
Any idea about this? ;-)

bo zhaobo <bz...@gmail.com> 于2022年8月23日周二 19:28写道：

> Hi Flinkers,
>
> In 2019, we raised an discussion in Flink about "ARM support for
> Flink"[1]. And we got so many helps and supports from Flink community about
> introducing a ARM CI system into Flink community, which is named
> "OpenLab"[2], and finally we create a full stack regression Flink tests on
> OpenLab ARM resources, then post a email to Flink Maillist about test
> result every day. We've been doing that for almost 2 years.
>
> But now, we are so apologized that OpenLab had been reached its EOL. We
> had to close it last month. So for sustaining the existing ARM CI still
> work on Flink Community and helping contributors to verify their code on
> ARM. We decide that we *will donate some ARM resources(Virtual Machines) *into
> Flink community to make this happen.
>
> And considering the existing Flink CICD had been moved to Azure Pipeline,
> and doesn't use github action.
>
> Now what we can provide are *ONLY *ARM resources(Virtual Machines) from
> us, so we think *Flink community is the right role to decide how to use
> them.*
> We only give several suggestions here:
> 1. Github action self-hosted Machines(Integrated with our ARM resources)
> 2. Azure Pipeline self-hosted Machines(Integrated with our ARM resources)
> 3. Any ideas from Flinkers?
>
> If community accepts our ARM resources and want to integrate with existing
> CICD in any way, please feel free to ping me about the quota(CPU nums,
> Memory&Disk size and so on) of VM we need to donate.
>
> Thank you very much.
>
> BR
>
> Bo Zhao
>
> [1] https://www.mail-archive.com/dev@flink.apache.org/msg27054.html
> [2] https://openlabtesting.org/
>