You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "carlthome (via GitHub)" <gi...@apache.org> on 2023/05/31 14:58:51 UTC
[GitHub] [beam] carlthome commented on pull request #16658: [BEAM-12792] Install pipeline dependencies to temporary venv
carlthome commented on PR #16658:
URL: https://github.com/apache/beam/pull/16658#issuecomment-1570401595
We're also affected by this on Dataflow when upgrading our Python container image from copying in Apache Beam 2.43 to now 2.46. Weirdly, python3-venv is installed but still seeing an infinite loop with
```
Failed to initialize Python venv.
```
until the Dataflow job times out.
This is the image contents as per `syft`:
```tsv
NAME VERSION TYPE
Markdown 3.4.3 python
MarkupSafe 2.1.2 python
Pillow 9.5.0 python
PyYAML 6.0 python
Werkzeug 2.3.4 python
absl-py 1.4.0 python
adduser 3.118ubuntu5 deb
apache-beam 2.46.0 python
apt 2.4.9 deb
array-record 0.2.0 python
astunparse 1.6.3 python
audioread 3.0.0 python
base-files 12ubuntu4.3 deb
base-passwd 3.5.52build1 deb
bash 5.1-6ubuntu1 deb
bsdutils 1:2.37.2-4ubuntu3 deb
ca-certificates 20230311ubuntu0.22.04.1 deb
cachetools 4.2.4 python
certifi 2023.5.7 python
cffi 1.15.1 python
charset-normalizer 3.1.0 python
click 8.1.3 python
cloudpickle 2.2.1 python
coreutils 8.32-4.1ubuntu1 deb
crcmod 1.7 python
dash 0.5.11+git20210903+057cd650a4ed-3build1 deb
db-dtypes 1.1.1 python
debconf 1.5.79ubuntu1 deb
debianutils 5.5-1ubuntu2 deb
decorator 5.1.1 python
diffutils 1:3.8-0ubuntu2 deb
dill 0.3.1.1 python
dm-tree 0.1.8 python
docopt 0.6.2 python
dpkg 1.21.1ubuntu2.1 deb
e2fsprogs 1.46.5-2ubuntu1.1 deb
etils 1.3.0 python
fastavro 1.7.4 python
fasteners 0.18 python
ffmpeg 7:4.4.2-0ubuntu0.22.04.1 deb
filelock 3.12.0 python
findutils 4.8.0-1ubuntu3 deb
flatbuffers 23.5.26 python
fontconfig 2.13.1-4.2ubuntu5 deb
fontconfig-config 2.13.1-4.2ubuntu5 deb
fonts-dejavu-core 2.37-2build1 deb
fsspec 2023.5.0 python
gast 0.4.0 python
gcc-12-base 12.1.0-2ubuntu1~22.04 deb
github.com/apache/beam/sdks/v2 (devel) go-module
github.com/golang/protobuf v1.5.2 go-module
golang.org/x/net v0.5.0 go-module
golang.org/x/sys v0.5.0 go-module
golang.org/x/text v0.7.0 go-module
google-api-core 2.11.0 python
google-apitools 0.5.31 python
google-auth 2.19.0 python
google-auth-httplib2 0.1.0 python
google-auth-oauthlib 1.0.0 python
google-cloud-bigquery 3.10.0 python
google-cloud-bigquery-storage 2.16.2 python
google-cloud-bigtable 1.7.3 python
google-cloud-core 2.3.2 python
google-cloud-datastore 1.15.5 python
google-cloud-dlp 3.12.1 python
google-cloud-language 1.3.2 python
google-cloud-pubsub 2.17.1 python
google-cloud-pubsublite 1.8.2 python
google-cloud-recommendations-ai 0.7.1 python
google-cloud-spanner 3.35.0 python
google-cloud-storage 2.8.0 python
google-cloud-videointelligence 1.16.3 python
google-cloud-vision 3.4.1 python
google-crc32c 1.5.0 python
google-pasta 0.2.0 python
google-resumable-media 2.5.0 python
google.golang.org/genproto v0.0.0-20230110181048-76db0878b65f go-module
google.golang.org/grpc v1.52.3 go-module
google.golang.org/protobuf v1.28.1 go-module
googleapis-common-protos 1.59.0 python
gpgv 2.2.27-3ubuntu2.1 deb
grep 3.7-1build1 deb
grpc-google-iam-v1 0.12.6 python
grpcio 1.54.2 python
grpcio-status 1.48.2 python
gzip 1.10-4ubuntu4.1 deb
h5py 3.8.0 python
hdfs 2.7.0 python
hostname 3.23ubuntu2 deb
httplib2 0.21.0 python
huggingface-hub 0.14.1 python
idna 3.4 python
importlib-resources 5.12.0 python
init-system-helpers 1.62 deb
jax 0.4.10 python
joblib 1.2.0 python
keras 2.12.0 python
libacl1 2.3.1-1 deb
libaom3 3.3.0-1 deb
libapparmor1 3.0.4-2ubuntu2.2 deb
libapt-pkg6.0 2.4.9 deb
libasound2 1.2.6.1-1ubuntu1 deb
libasound2-data 1.2.6.1-1ubuntu1 deb
libass9 1:0.15.2-1 deb
libasyncns0 0.8-6build2 deb
libattr1 1:2.5.1-1build1 deb
libaudit-common 1:3.0.7-1build1 deb
libaudit1 1:3.0.7-1build1 deb
libavc1394-0 0.5.4-5build2 deb
libavcodec58 7:4.4.2-0ubuntu0.22.04.1 deb
libavdevice58 7:4.4.2-0ubuntu0.22.04.1 deb
libavfilter7 7:4.4.2-0ubuntu0.22.04.1 deb
libavformat58 7:4.4.2-0ubuntu0.22.04.1 deb
libavutil56 7:4.4.2-0ubuntu0.22.04.1 deb
libblas3 3.10.0-2ubuntu1 deb
libblkid1 2.37.2-4ubuntu3 deb
libbluray2 1:1.3.1-1 deb
libbrotli1 1.0.9-2build6 deb
libbs2b0 3.1.0+dfsg-2.2build1 deb
libbsd0 0.11.5-1 deb
libbz2-1.0 1.0.8-5build1 deb
libc-bin 2.35-0ubuntu3.1 deb
libc6 2.35-0ubuntu3.1 deb
libcaca0 0.99.beta19-2.2ubuntu4 deb
libcairo-gobject2 1.16.0-5ubuntu2 deb
libcairo2 1.16.0-5ubuntu2 deb
libcap-ng0 0.7.9-2.2build3 deb
libcap2 1:2.44-1build3 deb
libcdio-cdda2 10.2+2.0.0-1build3 deb
libcdio-paranoia2 10.2+2.0.0-1build3 deb
libcdio19 2.1.0-3build1 deb
libchromaprint1 1.5.1-2 deb
libclang 16.0.0 python
libcodec2-1.0 1.0.1-3 deb
libcom-err2 1.46.5-2ubuntu1.1 deb
libcrypt1 1:4.4.27-1 deb
libdatrie1 0.2.13-2 deb
libdav1d5 0.9.2-1 deb
libdb5.3 5.3.28+dfsg1-0.8ubuntu3 deb
libdbus-1-3 1.12.20-2ubuntu4.1 deb
libdc1394-25 2.2.6-4 deb
libdebconfclient0 0.261ubuntu1 deb
libdecor-0-0 0.1.0-3build1 deb
libdeflate0 1.10-2 deb
libdrm-amdgpu1 2.4.113-2~ubuntu0.22.04.1 deb
libdrm-common 2.4.113-2~ubuntu0.22.04.1 deb
libdrm-nouveau2 2.4.113-2~ubuntu0.22.04.1 deb
libdrm-radeon1 2.4.113-2~ubuntu0.22.04.1 deb
libdrm2 2.4.113-2~ubuntu0.22.04.1 deb
libedit2 3.1-20210910-1build1 deb
libelf1 0.186-1build1 deb
libexpat1 2.4.7-1ubuntu0.2 deb
libext2fs2 1.46.5-2ubuntu1.1 deb
libffi8 3.4.2-4 deb
libflac8 1.3.3-2ubuntu0.1 deb
libflite1 2.2-3 deb
libfontconfig1 2.13.1-4.2ubuntu5 deb
libfreetype6 2.11.1+dfsg-1ubuntu0.2 deb
libfribidi0 1.0.8-2ubuntu3.1 deb
libgbm1 22.2.5-0ubuntu0.1~22.04.1 deb
libgcc-s1 12.1.0-2ubuntu1~22.04 deb
libgcrypt20 1.9.4-3ubuntu3 deb
libgdk-pixbuf-2.0-0 2.42.8+dfsg-1ubuntu0.2 deb
libgdk-pixbuf2.0-common 2.42.8+dfsg-1ubuntu0.2 deb
libgfortran5 12.1.0-2ubuntu1~22.04 deb
libgl1 1.4.0-1 deb
libgl1-mesa-dri 22.2.5-0ubuntu0.1~22.04.1 deb
libglapi-mesa 22.2.5-0ubuntu0.1~22.04.1 deb
libglib2.0-0 2.72.4-0ubuntu2 deb
libglvnd0 1.4.0-1 deb
libglx-mesa0 22.2.5-0ubuntu0.1~22.04.1 deb
libglx0 1.4.0-1 deb
libgme0 0.6.3-2 deb
libgmp10 2:6.2.1+dfsg-3ubuntu1 deb
libgnutls30 3.7.3-4ubuntu1.2 deb
libgomp1 12.1.0-2ubuntu1~22.04 deb
libgpg-error0 1.43-3 deb
libgraphite2-3 1.3.14-1build2 deb
libgsm1 1.0.19-1 deb
libgssapi-krb5-2 1.19.2-2ubuntu0.1 deb
libharfbuzz0b 2.7.4-1ubuntu3.1 deb
libhogweed6 3.7.3-1build2 deb
libicu70 70.1-2 deb
libidn2-0 2.3.2-2build1 deb
libiec61883-0 1.2.0-4build3 deb
libjack-jackd2-0 1.9.20~dfsg-1 deb
libjbig0 2.1-3.1ubuntu0.22.04.1 deb
libjpeg-turbo8 2.1.2-0ubuntu1 deb
libjpeg8 8c-2ubuntu10 deb
libk5crypto3 1.19.2-2ubuntu0.1 deb
libkeyutils1 1.6.1-2ubuntu3 deb
libkrb5-3 1.19.2-2ubuntu0.1 deb
libkrb5support0 1.19.2-2ubuntu0.1 deb
liblapack3 3.10.0-2ubuntu1 deb
liblilv-0-0 0.24.12-2 deb
libllvm15 1:15.0.7-0ubuntu0.22.04.1 deb
liblz4-1 1.9.3-2build2 deb
liblzma5 5.2.5-2ubuntu1 deb
libmd0 1.0.4-1build1 deb
libmfx1 22.3.0-1 deb
libmount1 2.37.2-4ubuntu3 deb
libmp3lame0 3.100-3build2 deb
libmpdec3 2.5.1-2build2 deb
libmpg123-0 1.29.3-1build1 deb
libmysofa1 1.2.1~dfsg0-1 deb
libncurses6 6.3-2 deb
libncursesw6 6.3-2 deb
libnettle8 3.7.3-1build2 deb
libnorm1 1.5.9+dfsg-2 deb
libnsl2 1.3.0-2build2 deb
libnuma1 2.0.14-3ubuntu2 deb
libogg0 1.3.5-0ubuntu3 deb
libopenal-data 1:1.19.1-2build3 deb
libopenal1 1:1.19.1-2build3 deb
libopenjp2-7 2.4.0-6 deb
libopenmpt0 0.6.1-1 deb
libopus0 1.3.1-0.1build2 deb
libp11-kit0 0.24.0-6build1 deb
libpam-modules 1.4.0-11ubuntu2.3 deb
libpam-modules-bin 1.4.0-11ubuntu2.3 deb
libpam-runtime 1.4.0-11ubuntu2.3 deb
libpam0g 1.4.0-11ubuntu2.3 deb
libpango-1.0-0 1.50.6+ds-2ubuntu1 deb
libpangocairo-1.0-0 1.50.6+ds-2ubuntu1 deb
libpangoft2-1.0-0 1.50.6+ds-2ubuntu1 deb
libpcre2-8-0 10.39-3ubuntu0.1 deb
libpcre3 2:8.39-13ubuntu0.22.04.1 deb
libpgm-5.3-0 5.3.128~dfsg-2 deb
libpixman-1-0 0.40.0-1ubuntu0.22.04.1 deb
libpng16-16 1.6.37-3build5 deb
libpocketsphinx3 0.8.0+real5prealpha+1-14ubuntu1 deb
libpostproc55 7:4.4.2-0ubuntu0.22.04.1 deb
libprocps8 2:3.3.17-6ubuntu2 deb
libpulse0 1:15.99.1+dfsg1-1ubuntu2.1 deb
libpython3-stdlib 3.10.6-1~22.04 deb
libpython3.10-minimal 3.10.6-1~22.04.2ubuntu1 deb
libpython3.10-stdlib 3.10.6-1~22.04.2ubuntu1 deb
libquadmath0 12.1.0-2ubuntu1~22.04 deb
librabbitmq4 0.10.0-1ubuntu2 deb
libraw1394-11 2.1.2-2build2 deb
libreadline8 8.1.2-1 deb
librosa 0.8.0 python
librsvg2-2 2.52.5+dfsg-3 deb
librubberband2 2.0.0-2 deb
libsamplerate0 0.2.2-1build1 deb
libsdl2-2.0-0 2.0.20+dfsg-2ubuntu1.22.04.1 deb
libseccomp2 2.5.3-2ubuntu2 deb
libselinux1 3.3-1build2 deb
libsemanage-common 3.3-1build2 deb
libsemanage2 3.3-1build2 deb
libsensors-config 1:3.6.0-7ubuntu1 deb
libsensors5 1:3.6.0-7ubuntu1 deb
libsepol2 3.3-1build1 deb
libserd-0-0 0.30.10-2 deb
libshine3 3.1.1-2 deb
libslang2 2.3.2-5build4 deb
libsmartcols1 2.37.2-4ubuntu3 deb
libsnappy1v5 1.1.8-1build3 deb
libsndfile1 1.0.31-2build1 deb
libsndio7.0 1.8.1-1.1 deb
libsodium23 1.0.18-1build2 deb
libsord-0-0 0.16.8-2 deb
libsoxr0 0.1.3-4build2 deb
libspeex1 1.2~rc1.2-1.1ubuntu3 deb
libsphinxbase3 0.8+5prealpha+1-13build1 deb
libsqlite3-0 3.37.2-2ubuntu0.1 deb
libsratom-0-0 0.6.8-1 deb
libsrt1.4-gnutls 1.4.4-4 deb
libss2 1.46.5-2ubuntu1.1 deb
libssh-gcrypt-4 0.9.6-2build1 deb
libssl3 3.0.2-0ubuntu1.9 deb
libstdc++6 12.1.0-2ubuntu1~22.04 deb
libswresample3 7:4.4.2-0ubuntu0.22.04.1 deb
libswscale5 7:4.4.2-0ubuntu0.22.04.1 deb
libsystemd0 249.11-0ubuntu3.9 deb
libtasn1-6 4.18.0-4build1 deb
libthai-data 0.1.29-1build1 deb
libthai0 0.1.29-1build1 deb
libtheora0 1.1.1+dfsg.1-15ubuntu4 deb
libtiff5 4.3.0-6ubuntu0.4 deb
libtinfo6 6.3-2 deb
libtirpc-common 1.3.2-2ubuntu0.1 deb
libtirpc3 1.3.2-2ubuntu0.1 deb
libtwolame0 0.4.0-2build2 deb
libudev1 249.11-0ubuntu3.9 deb
libudfread0 1.1.2-1 deb
libunistring2 1.0-1 deb
libusb-1.0-0 2:1.0.25-1ubuntu2 deb
libuuid1 2.37.2-4ubuntu3 deb
libva-drm2 2.14.0-1 deb
libva-x11-2 2.14.0-1 deb
libva2 2.14.0-1 deb
libvdpau1 1.4-3build2 deb
libvidstab1.1 1.1.0-2 deb
libvorbis0a 1.3.7-1build2 deb
libvorbisenc2 1.3.7-1build2 deb
libvorbisfile3 1.3.7-1build2 deb
libvpx7 1.11.0-2ubuntu2 deb
libwayland-client0 1.20.0-1ubuntu0.1 deb
libwayland-cursor0 1.20.0-1ubuntu0.1 deb
libwayland-egl1 1.20.0-1ubuntu0.1 deb
libwayland-server0 1.20.0-1ubuntu0.1 deb
libwebp7 1.2.2-2ubuntu0.22.04.1 deb
libwebpmux3 1.2.2-2ubuntu0.22.04.1 deb
libx11-6 2:1.7.5-1 deb
libx11-data 2:1.7.5-1 deb
libx11-xcb1 2:1.7.5-1 deb
libx264-163 2:0.163.3060+git5db6aa6-2build1 deb
libx265-199 3.5-2 deb
libxau6 1:1.0.9-1build5 deb
libxcb-dri2-0 1.14-3ubuntu3 deb
libxcb-dri3-0 1.14-3ubuntu3 deb
libxcb-glx0 1.14-3ubuntu3 deb
libxcb-present0 1.14-3ubuntu3 deb
libxcb-render0 1.14-3ubuntu3 deb
libxcb-shape0 1.14-3ubuntu3 deb
libxcb-shm0 1.14-3ubuntu3 deb
libxcb-sync1 1.14-3ubuntu3 deb
libxcb-xfixes0 1.14-3ubuntu3 deb
libxcb1 1.14-3ubuntu3 deb
libxcursor1 1:1.2.0-2build4 deb
libxdmcp6 1:1.1.3-0ubuntu5 deb
libxext6 2:1.3.4-1build1 deb
libxfixes3 1:6.0.0-1 deb
libxi6 2:1.8-1build1 deb
libxinerama1 2:1.1.4-3 deb
libxkbcommon0 1.4.0-1 deb
libxml2 2.9.13+dfsg-1ubuntu0.3 deb
libxrandr2 2:1.5.2-1build1 deb
libxrender1 1:0.9.10-1build4 deb
libxshmfence1 1.3-1build4 deb
libxss1 1:1.2.3-1build2 deb
libxv1 2:1.0.11-1build2 deb
libxvidcore4 2:1.3.7-1 deb
libxxf86vm1 1:1.1.4-1build3 deb
libxxhash0 0.8.1-1 deb
libzimg2 3.0.3+ds1-1 deb
libzmq5 4.3.4-2 deb
libzstd1 1.4.8+dfsg-3build1 deb
libzvbi-common 0.2.35-19 deb
libzvbi0 0.2.35-19 deb
llvmlite 0.40.0 python
login 1:4.8.1-2ubuntu2.1 deb
logsave 1.46.5-2ubuntu1.1 deb
lsb-base 11.1.0ubuntu4 deb
mawk 1.3.4.20200120-3 deb
media-types 7.0.0 deb
ml-dtypes 0.1.0 python
mount 2.37.2-4ubuntu3 deb
ncurses-base 6.3-2 deb
ncurses-bin 6.3-2 deb
numba 0.57.0 python
numpy 1.23.5 python
oauth2client 4.1.3 python
oauthlib 3.2.2 python
objsize 0.6.1 python
ocl-icd-libopencl1 2.2.14-3 deb
openssl 3.0.2-0ubuntu1.9 deb
opt-einsum 3.3.0 python
orjson 3.8.14 python
overrides 6.5.0 python
packaging 23.1 python
pandas 2.0.1 python
pandas-gbq 0.19.1 python
passwd 1:4.8.1-2ubuntu2.1 deb
perl-base 5.34.0-3ubuntu1.1 deb
pip 22.0.2 python
platformdirs 3.5.1 python
pooch 1.7.0 python
procps 2:3.3.17-6ubuntu2 deb
promise 2.3 python
proto-plus 1.22.2 python
protobuf 3.20.3 python
psutil 5.9.5 python
pyarrow 9.0.0 python
pyasn1 0.5.0 python
pyasn1-modules 0.3.0 python
pycparser 2.21 python
pydata-google-auth 1.8.0 python
pydot 1.4.2 python
pymongo 3.13.0 python
pyparsing 3.0.9 python
python-dateutil 2.8.2 python
python3 3.10.6-1~22.04 deb
python3-distutils 3.10.6-1~22.04 deb
python3-lib2to3 3.10.6-1~22.04 deb
python3-minimal 3.10.6-1~22.04 deb
python3-pip 22.0.2+dfsg-1ubuntu0.2 deb
python3-pip-whl 22.0.2+dfsg-1ubuntu0.2 deb
python3-pkg-resources 59.6.0-1.2ubuntu0.22.04.1 deb
python3-setuptools 59.6.0-1.2ubuntu0.22.04.1 deb
python3-setuptools-whl 59.6.0-1.2ubuntu0.22.04.1 deb
python3-venv 3.10.6-1~22.04 deb
python3-wheel 0.37.1-2ubuntu0.22.04.1 deb
python3.10 3.10.6-1~22.04.2ubuntu1 deb
python3.10-minimal 3.10.6-1~22.04.2ubuntu1 deb
python3.10-venv 3.10.6-1~22.04.2ubuntu1 deb
pytz 2023.3 python
readline-common 8.1.2-1 deb
regex 2023.5.5 python
requests 2.31.0 python
requests-oauthlib 1.3.1 python
resampy 0.4.2 python
rsa 4.9 python
scikit-learn 1.2.2 python
scipy 1.10.1 python
sed 4.8-1ubuntu2 deb
sensible-utils 0.0.17 deb
setuptools 59.6.0 python
shared-mime-info 2.1-2 deb
six 1.16.0 python
soundfile 0.12.1 python
sqlparse 0.4.4 python
sysvinit-utils 3.01-1ubuntu1 deb
tar 1.34+dfsg-1ubuntu0.1.22.04.1 deb
tensorboard 2.12.3 python
tensorboard-data-server 0.7.0 python
tensorflow 2.12.0 python
tensorflow-datasets 4.9.2 python
tensorflow-estimator 2.12.0 python
tensorflow-hub 0.13.0 python
tensorflow-io-gcs-filesystem 0.32.0 python
tensorflow-metadata 1.13.1 python
termcolor 2.3.0 python
threadpoolctl 3.1.0 python
tokenizers 0.13.3 python
toml 0.10.2 python
tqdm 4.65.0 python
transformers 4.28.1 python
typing_extensions 4.6.2 python
tzdata 2023.3 python
ubuntu-keyring 2021.03.26 deb
ucf 3.0043 deb
urllib3 1.26.16 python
usrmerge 25ubuntu2 deb
util-linux 2.37.2-4ubuntu3 deb
wheel 0.37.1 python
wrapt 1.14.1 python
x11-common 1:7.7+23ubuntu2 deb
xkb-data 2.33-1 deb
zipp 3.15.0 python
zlib1g 1:1.2.11.dfsg-2ubuntu9.2 deb
zstandard 0.21.0 python
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org