You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@devlake.apache.org by Ji Bin <ji...@apache.org> on 2023/01/03 05:20:53 UTC
Solution about support arm64 release
Hi, team
Happy New Year.
Recently, I'm working on some PoC for providing an arm64-based devlake
release, mentioned in [Question][Docker Image] ARM version · Issue #4016 ·
apache/incubator-devlake (github.com)
<https://github.com/apache/incubator-devlake/issues/4016>.
Currently, the main branch uses alpine as a base image, as alpine does not
provide perfect cross-compile toolchains, as we need libgit2, which
requires CGO in go and also needs to compile it from source (1.3
currently). And Debian could provide perfect cross-compile toolchains. I
suggest we use Debian as the base image in an official release.
Before that PR starting, I want here more comments and suggestions from
teams, as this change will impact the base image we select.
In this PoC, feat: try build multi-arch docker images for amd64 and arm64 ·
matrixji/incubator-devlake@e07f69b (github.com)
<https://github.com/matrixji/incubator-devlake/commit/e07f69b62f1edc96bf4aae2ad84ba8abd8008a3c>
I
summarise the changes below:
- Use docker's action to build and push images, and describe all things in
one Dockerfile, maybe in the final PR, It could be separated into several
images like builder-image, base-image
- We will use buildx for build docker images, which requires BUILD_KIT
enabled while the developer builds images locally.
- Use Debian as the base image, as it provides perfect cross-compiling
toolchains. Which will increase the final image's size ( <100MB), from
500MB+ to 600MB+.
Waiting for your comments about it.
BRs, Ji Bin.
Re: Solution about support arm64 release
Posted by Yingchu Chen <Yi...@merico.dev.INVALID>.
I checked both images:
multi-arch: [cid:9a018da5-3aee-4b69-aa57-500e68bf8a88]
original:
[cid:80f482b5-fe4f-4045-8a7b-6b9c79ad0b58]
For arm64, the size increased 110mb, which increased 140mb for amd64. his is acceptable.
________________________________
From: Klesh Wong <kl...@apache.org>
Sent: Tuesday, January 3, 2023 15:39
To: dev@devlake.apache.org <de...@devlake.apache.org>
Subject: Re: Solution about support arm64 release
Hi, Ji Bin.
Thanks for the hard work.
The description of the plan and impact are very thorough, appreciated!
I think it is acceptable to trade "Multi-Arch Support" with "<100MB
image size increase".
Thanks
Klesh Wong
On 23/01/03 01:20PM, Ji Bin wrote:
> Hi, team
>
> Happy New Year.
>
> Recently, I'm working on some PoC for providing an arm64-based devlake
> release, mentioned in [Question][Docker Image] ARM version · Issue #4016 ·
> apache/incubator-devlake (github.com)
> <https://github.com/apache/incubator-devlake/issues/4016>.
> Currently, the main branch uses alpine as a base image, as alpine does not
> provide perfect cross-compile toolchains, as we need libgit2, which
> requires CGO in go and also needs to compile it from source (1.3
> currently). And Debian could provide perfect cross-compile toolchains. I
> suggest we use Debian as the base image in an official release.
>
> Before that PR starting, I want here more comments and suggestions from
> teams, as this change will impact the base image we select.
>
> In this PoC, feat: try build multi-arch docker images for amd64 and arm64 ·
> matrixji/incubator-devlake@e07f69b (github.com)
> <https://github.com/matrixji/incubator-devlake/commit/e07f69b62f1edc96bf4aae2ad84ba8abd8008a3c>
> I
> summarise the changes below:
> - Use docker's action to build and push images, and describe all things in
> one Dockerfile, maybe in the final PR, It could be separated into several
> images like builder-image, base-image
> - We will use buildx for build docker images, which requires BUILD_KIT
> enabled while the developer builds images locally.
> - Use Debian as the base image, as it provides perfect cross-compiling
> toolchains. Which will increase the final image's size ( <100MB), from
> 500MB+ to 600MB+.
>
> Waiting for your comments about it.
>
> BRs, Ji Bin.
Re: Solution about support arm64 release
Posted by "Matrix.Ji" <we...@gmail.com>.
Hi Hezheng,
For the cross-compile toolchains in Debian, it provides
`gcc-aarch64-linux-gnu`, and in Debian:arm64 distribution it also provides
`gcc-x86-64-linux-gnu`, which means we are able to build an arm64 builder
image for guys who use arm64 based device as developer machine, e.g. M1/2
based mac.
For the alpine, currently, in x86_64 distribution, it provides
`gcc-aarch64-none-elf`, and I met 2 problems:
- Previously I run the compiled lake binary it crashes
- And still, I could not find some package under alpine:arm64 to compile
86_64 binaries.
For ZhangLiang's comments, it seems not available for now, as Debian uses
glibc by default, while alpine uses musl. I guess there will be some
compatibility problems.
Still, the solution for using alpine is worth trying, I'll continue to have
more investigation on it.
BRs, Ji Bin
Hezheng Yin <yi...@gmail.com> 于2023年1月4日周三 07:03写道：
> Hi Bin,
>
> Thanks for starting this thread. The slight increase in the image size
> isn't a significant concern. Out of curiosity, what cross-compile toolchain
> does alpine lack while Debian provides?
>
> Best,
> Hezheng
>
>
> On Tue, Jan 3, 2023 at 12:53 AM Liang Zhang <liang.zhang@merico.dev
> .invalid>
> wrote:
>
> >
> > Hi, Ji Bin
> >
> > I think it does not matter if the image size increases by 20%. I have a
> > question about the cross-compile, could we just change the builder from
> > alpine to Debian and keep the rest?
> >
> > Best Regards
> >
> > Zhang Liang
>
Re: Solution about support arm64 release
Posted by Hezheng Yin <yi...@gmail.com>.
Hi Bin,
Thanks for starting this thread. The slight increase in the image size
isn't a significant concern. Out of curiosity, what cross-compile toolchain
does alpine lack while Debian provides?
Best,
Hezheng
On Tue, Jan 3, 2023 at 12:53 AM Liang Zhang <li...@merico.dev.invalid>
wrote:
>
> Hi, Ji Bin
>
> I think it does not matter if the image size increases by 20%. I have a
> question about the cross-compile, could we just change the builder from
> alpine to Debian and keep the rest?
>
> Best Regards
>
> Zhang Liang
Re: Solution about support arm64 release
Posted by Liang Zhang <li...@merico.dev.INVALID>.
Hi, Ji Bin
I think it does not matter if the image size increases by 20%. I have a question about the cross-compile, could we just change the builder from alpine to Debian and keep the rest?
Best Regards
Zhang Liang
Re: Solution about support arm64 release
Posted by Klesh Wong <kl...@apache.org>.
Hi, Ji Bin.
Thanks for the hard work.
The description of the plan and impact are very thorough, appreciated!
I think it is acceptable to trade "Multi-Arch Support" with "<100MB
image size increase".
Thanks
Klesh Wong
On 23/01/03 01:20PM, Ji Bin wrote:
> Hi, team
>
> Happy New Year.
>
> Recently, I'm working on some PoC for providing an arm64-based devlake
> release, mentioned in [Question][Docker Image] ARM version · Issue #4016 ·
> apache/incubator-devlake (github.com)
> <https://github.com/apache/incubator-devlake/issues/4016>.
> Currently, the main branch uses alpine as a base image, as alpine does not
> provide perfect cross-compile toolchains, as we need libgit2, which
> requires CGO in go and also needs to compile it from source (1.3
> currently). And Debian could provide perfect cross-compile toolchains. I
> suggest we use Debian as the base image in an official release.
>
> Before that PR starting, I want here more comments and suggestions from
> teams, as this change will impact the base image we select.
>
> In this PoC, feat: try build multi-arch docker images for amd64 and arm64 ·
> matrixji/incubator-devlake@e07f69b (github.com)
> <https://github.com/matrixji/incubator-devlake/commit/e07f69b62f1edc96bf4aae2ad84ba8abd8008a3c>
> I
> summarise the changes below:
> - Use docker's action to build and push images, and describe all things in
> one Dockerfile, maybe in the final PR, It could be separated into several
> images like builder-image, base-image
> - We will use buildx for build docker images, which requires BUILD_KIT
> enabled while the developer builds images locally.
> - Use Debian as the base image, as it provides perfect cross-compiling
> toolchains. Which will increase the final image's size ( <100MB), from
> 500MB+ to 600MB+.
>
> Waiting for your comments about it.
>
> BRs, Ji Bin.