You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "acgray (via GitHub)" <gi...@apache.org> on 2023/11/07 22:29:15 UTC
[I] Inconsistent architecture for generated images [superset]
acgray opened a new issue, #25899:
URL: https://github.com/apache/superset/issues/25899
The docker build script (`scripts/docker_build_push.sh`) appears to have inconsistent architecture specifications for the different images produced.
The 'dockerize' and 'websocket' images are built for both amd64 and arm64 but the others seem to built for amd64 only.
Is there a reason for this or can all images be built for both platforms?
Currently we are unable to run Superset on our arm64 servers as a result and are building our own image privately.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Inconsistent architecture for generated images [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on issue #25899:
URL: https://github.com/apache/superset/issues/25899#issuecomment-1990131780
I think this has largely been resolved by recent docker-related cleanup efforts. Can anyone confirm?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Inconsistent architecture for generated images [superset]
Posted by "sfirke (via GitHub)" <gi...@apache.org>.
sfirke commented on issue #25899:
URL: https://github.com/apache/superset/issues/25899#issuecomment-1810807146
Hi, this is a good question. I'm not sure why Superset some images would be built for arm64 and not others. I believe (not positive) the project's stance currently is that arm64 is not "officially supported." That said, it could be, and what it would take is in part what you are offering: someone to care for the arm64 builds, keep things working, etc. The Kubernetes Helm chart is in the same situation.
It seems like there is broad demand for arm64 builds, you're not alone there (see https://github.com/apache/superset/issues/25433 and the linked discussion there). I think yes, submit a PR? Especially if it won't be a ton of work, it may attract more attention there.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Inconsistent architecture for generated images [superset]
Posted by "iFrozenPhoenix (via GitHub)" <gi...@apache.org>.
iFrozenPhoenix commented on issue #25899:
URL: https://github.com/apache/superset/issues/25899#issuecomment-1871934945
@adimyth yeah this sounds familiar... I had the same problem at the beginning. This is very likely an oom kill. The build process needs about 4 to 5 GB of free RAM (available under free -m).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Inconsistent architecture for generated images [superset]
Posted by "adimyth (via GitHub)" <gi...@apache.org>.
adimyth commented on issue #25899:
URL: https://github.com/apache/superset/issues/25899#issuecomment-1872128911
Thanks! Beefing up the EC2 instance where I was building the image helped!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Inconsistent architecture for generated images [superset]
Posted by "iFrozenPhoenix (via GitHub)" <gi...@apache.org>.
iFrozenPhoenix commented on issue #25899:
URL: https://github.com/apache/superset/issues/25899#issuecomment-1825956578
@sfirke, @acgray on my site is the same that i currently build the image myself. There is no change needed, only the build arg must be specified to build it for arm64. The way we do it is to modify the Dockerfile so that we can build it directly for both platforms in a multilayer image. So omitting 1 buildarg is everything what is curently necessary to enable multi arch builds.
The relevant line is [here](https://github.com/apache/superset/blob/aad67e43dbabadad9a5e4accb29ecefb39315f6e/Dockerfile#L24)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Inconsistent architecture for generated images [superset]
Posted by "adimyth (via GitHub)" <gi...@apache.org>.
adimyth commented on issue #25899:
URL: https://github.com/apache/superset/issues/25899#issuecomment-1871891482
@iFrozenPhoenix I tried building the image on an ARM machine with the following steps, it failed for me.
1. Check the architecture of the machine
```bash
dpkg --print-architecture
arm64
```
```bash
uname -m
aarch64
```
2. Clone the repository and set the build platform to arm64
```bash
git clone https://github.com/apache/superset.git
cd superset
BUILDPLATFORM=arm64
```
3. Build the docker image
```bash
docker build -t superset-arm64 .
```
Error log:
```bash
[+] Building 125.7s (14/23) docker:default
=> [internal] load build definition from Dockerfile 6.5s
=> => transferring dockerfile: 5.78kB 0.0s
=> [internal] load .dockerignore 6.5s
=> => transferring context: 1.26kB 0.0s
=> [internal] load metadata for docker.io/library/python:3.9-slim-bookworm 1.4s
=> [internal] load metadata for docker.io/library/node:16-bookworm-slim 1.4s
=> [superset-node 1/7] FROM docker.io/library/node:16-bookworm-slim@sha256:89061680359f5d9103d24f0066e2c96a8018c24d5c4fe8eb8d59d16dd4ab64ba 0.0s
=> [internal] load build context 0.7s
=> => transferring context: 463.22kB 0.7s
=> [lean 1/10] FROM docker.io/library/python:3.9-slim-bookworm@sha256:96be08c44307e781fd9ce8e05b49c969b4cb902ec23594f904739c58da3a09ed 0.0s
=> CACHED [superset-node 2/7] RUN apt-get update -qq && apt-get install -yqq --no-install-recommends build-essential python3 0.0s
=> CACHED [superset-node 3/7] WORKDIR /app/superset-frontend 0.0s
=> CACHED [superset-node 4/7] RUN --mount=type=bind,target=/frontend-mem-nag.sh,src=./docker/frontend-mem-nag.sh /frontend-mem-nag.sh 0.0s
=> CACHED [lean 2/10] WORKDIR /app 0.0s
=> CACHED [lean 3/10] RUN mkdir -p /app/pythonpath superset/static superset-frontend apache_superset.egg-info requirements && useradd --user-group -d /app/superset_home -m --no-log-init --shell /bin/bash superset && apt-get update -qq && apt-get install -yqq --no-install-recommends build- 0.0s
=> CACHED [lean 4/10] COPY --chown=superset:superset setup.py MANIFEST.in README.md ./ 0.0s
=> CACHED [lean 5/10] COPY --chown=superset:superset superset-frontend/package.json superset-frontend/ 0.0s
=> [superset-node 5/7] RUN --mount=type=bind,target=./package.json,src=./superset-frontend/package.json --mount=type=bind,target=./package-lock.json,src=./superset-frontend/package-lock.json npm ci 117.0s
=> => # npm WARN deprecated fontsource-fira-code@4.0.0: Package relocated. Please install and migrate to @fontsource/fira-code.
=> => # npm WARN deprecated puppeteer@10.2.0: < 18.1.0 is no longer supported
=> => # npm WARN deprecated @babel/polyfill@7.12.1: 🚨 This package has been deprecated in favor of separate inclusion of a polyfill and regenerator-runtime (when needed). See the @babel/polyfill docs (https://babeljs.io/docs/en/babel-polyfill) for more information.
=> => # npm WARN deprecated chokidar@2.1.8: Chokidar 2 does not receive security updates since 2019. Upgrade to chokidar 3 with 15x fewer dependencies
=> => # npm WARN deprecated debug@4.1.1: Debug versions >=3.2.0 <3.2.7 || >=4 <4.3.1 have a low-severity ReDos regression when used in a Node.js environment. It is recommended you upgrade to 3.2.7 or 4.3.1. (https://github.com/visionmedia/debug/issues/797)
=> => # npm WARN deprecated debug@4.1.1: Debug versions >=3.2.0 <3.2.7 || >=4 <4.3.1 have a low-severity ReDos regression when used in a Node.js environment. It is recommended you upgrade to 3.2.7 or 4.3.1. (https://github.com/visionmedia/debug/issues/797)
=> [lean 6/10] RUN --mount=type=bind,target=./requirements/local.txt,src=./requirements/local.txt --mount=type=bind,target=./requirements/development.txt,src=./requirements/development.txt --mount=type=bind,target=./requirements/base.txt,src=./requirements/base.txt --mount=type=cache,target 117.0s
=> => # Preparing metadata (setup.py): finished with status 'done'
=> => # Collecting xlrd==2.0.1
=> => # Using cached xlrd-2.0.1-py2.py3-none-any.whl (96 kB)
=> => # Requirement already satisfied: setuptools in /usr/local/lib/python3.9/site-packages (from celery==5.2.2->-r requirements/base.txt (line 43)) (58.1.0)
=> => # Building wheels for collected packages: apsw, bottleneck, cron-descriptor, func-timeout, pgsanity, pymeeus, python-geohash, shortid, wtforms-json, future, linear-tsv, mysqlclient, progress, pure-sasl, pydruid, python-ldap, thrift, unicodecsv, pyhive
=> => # Building wheel for apsw (pyproject.toml): started
```
And then it abruptly stopped with no followup logs or anything.
Did it work for you? If so, can you please share the steps you followed?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org