You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@trafficserver.apache.org by "bneradt (via GitHub)" <gi...@apache.org> on 2023/06/24 21:55:41 UTC
[GitHub] [trafficserver] bneradt opened a new pull request, #9910: Add back python3-devel check for autests
bneradt opened a new pull request, #9910:
URL: https://github.com/apache/trafficserver/pull/9910
It turns out that autest requires python3-config to be installed on the system in order to run. Without it, pipenv install for the tests/Pipfile fails for very non-obvious reasons. Adding python3-devel check back.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@trafficserver.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] jpeach commented on pull request #9910: Add back python3-devel check for autests
Posted by "jpeach (via GitHub)" <gi...@apache.org>.
jpeach commented on PR #9910:
URL: https://github.com/apache/trafficserver/pull/9910#issuecomment-1607053288
I think there's something a little more subtle going on, because I don't need `python3-devel` on my F38 machine:
```
[jpeach@sk1 tests]$ pipenv --rm
Removing virtualenv (/home/jpeach/.local/share/virtualenvs/tests-U2EyhSHq)...
[jpeach@sk1 tests]$ sudo dnf remove python3-devel
No match for argument: python3-devel
No packages marked for removal.
Dependencies resolved.
Nothing to do.
Complete!
[jpeach@sk1 tests]$ which python3-config
/usr/bin/which: no python3-config in (/home/jpeach/bin:/home/jpeach/go/bin:/home/jpeach/bin:/home/jpeach/go/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin)
[jpeach@sk1 tests]$ pipenv install
Creating a virtualenv for this project...
Pipfile: /home/jpeach/src/trafficserver/tests/Pipfile
Using /usr/bin/python3 (3.11.3) to create virtualenv...
⠸ Creating virtual environment...created virtual environment CPython3.11.3.final.0-64 in 156ms
creator Venv(dest=/home/jpeach/.local/share/virtualenvs/tests-U2EyhSHq, clear=False, no_vcs_ignore=False, global=False, describe=CPython3Posix)
seeder FromAppData(extra_search_dir=/usr/share/python-wheels,download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/jpeach/.local/share/virtualenv)
added seed packages: pip==22.3.1, setuptools==65.5.1, wheel==0.38.4
activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
✔ Successfully created virtual environment!
Virtualenv location: /home/jpeach/.local/share/virtualenvs/tests-U2EyhSHq
Installing dependencies from Pipfile.lock (5beb05)...
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@trafficserver.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] bneradt commented on pull request #9910: Add back python3-devel check for autests
Posted by "bneradt (via GitHub)" <gi...@apache.org>.
bneradt commented on PR #9910:
URL: https://github.com/apache/trafficserver/pull/9910#issuecomment-1605771354
@JosiahWI was correct with this comment:
https://github.com/apache/trafficserver/pull/9891#issuecomment-1600603785
The need for python3-devel can be demonstrated using our CI's fedora:38 docker image like so:
```sh
╰─➤ docker run --rm -it ci.trafficserver.apache.org/ats/fedora:38 /bin/bash
[root@a4b15951163d ~]# cd /var/tmp
[root@a4b15951163d tmp]# git clone https://github.com/apache/trafficserver.git
...
[root@a4b15951163d tmp]# cd trafficserver/tests/
# Remove python3-devel before installing the pipenv.
[root@a4b15951163d tests]# sudo dnf remove -y python3-devel
...
Removed:
python3-devel-3.11.3-2.fc38.aarch64 python3-packaging-23.0-1.fc38.noarch python3-rpm-generators-14-4.fc38.noarch
Complete!
# Now try to install. Notice that the autest dependencies cannot be built due to the lack of the Python3 dev header.
[root@a4b15951163d tests]# pipenv install
Creating a virtualenv for this project...
Pipfile: /var/tmp/trafficserver/tests/Pipfile
Using /usr/bin/python3 (3.11.3) to create virtualenv...
⠙ Creating virtual environment...created virtual environment CPython3.11.3.final.0-64 in 175ms
creator CPython3Posix(dest=/root/.local/share/virtualenvs/tests-HYg0op0j, clear=False, no_vcs_ignore=False, global=False)
seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/root/.local/share/virtualenv)
added seed packages: pip==23.1.2, setuptools==67.8.0, wheel==0.40.0
activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
✔ Successfully created virtual environment!
Virtualenv location: /root/.local/share/virtualenvs/tests-HYg0op0j
Pipfile.lock not found, creating...
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
✔ Success!
Locking [dev-packages] dependencies...
Building requirements...
Resolving dependencies...
✔ Success!
Updated Pipfile.lock (3094ee8b548fecf9988a8ccede4c1953ffb4925980fddb241c9f254eb85beb05)!
Installing dependencies from Pipfile.lock (5beb05)...
An error occurred while installing attrs==23.1.0 ; python_version >= '3.7' --hash=sha256:1f28b4522cdc2fb4256ac1a020c78acf9cba2c6b461ccd2c126f3aa8e8335d04 --hash=sha256:6279836d581513a26f1bf235f9acd333bc9115683f14f7e8fae46c98fc50e015! Will try again.
An error occurred while installing autest==1.10.2 --hash=sha256:ec26de605853e04425918d26073fa26d58bec9f785cb35542ed1a4ee2b111f73! Will try again.
...
# Now repeat with python3-devel installed
[root@a4b15951163d tests]# pipenv --rm
Removing virtualenv (/root/.local/share/virtualenvs/tests-HYg0op0j)...
[root@a4b15951163d tests]# sudo dnf install -y python3-devel
...
Installed:
python3-devel-3.11.3-2.fc38.aarch64 python3-packaging-23.0-1.fc38.noarch python3-rpm-generators-14-4.fc38.noarch
Complete!
# Now everything is happy.
[root@a4b15951163d tests]# pipenv install
Creating a virtualenv for this project...
Pipfile: /var/tmp/trafficserver/tests/Pipfile
Using /usr/bin/python3 (3.11.3) to create virtualenv...
⠋ Creating virtual environment...created virtual environment CPython3.11.3.final.0-64 in 88ms
creator CPython3Posix(dest=/root/.local/share/virtualenvs/tests-HYg0op0j, clear=False, no_vcs_ignore=False, global=False)
seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/root/.local/share/virtualenv)
added seed packages: pip==23.1.2, setuptools==67.8.0, wheel==0.40.0
activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
✔ Successfully created virtual environment!
Virtualenv location: /root/.local/share/virtualenvs/tests-HYg0op0j
Installing dependencies from Pipfile.lock (5beb05)...
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@trafficserver.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] bneradt merged pull request #9910: Add back python3-devel check for autests
Posted by "bneradt (via GitHub)" <gi...@apache.org>.
bneradt merged PR #9910:
URL: https://github.com/apache/trafficserver/pull/9910
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@trafficserver.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] bneradt commented on pull request #9910: Add back python3-devel check for autests
Posted by "bneradt (via GitHub)" <gi...@apache.org>.
bneradt commented on PR #9910:
URL: https://github.com/apache/trafficserver/pull/9910#issuecomment-1607952053
> I think there's something a little more subtle going on, because I don't need `python3-devel` on my F38 machine:
>
> ```
> [jpeach@sk1 tests]$ pipenv --rm
> Removing virtualenv (/home/jpeach/.local/share/virtualenvs/tests-U2EyhSHq)...
> [jpeach@sk1 tests]$ sudo dnf remove python3-devel
> No match for argument: python3-devel
> No packages marked for removal.
> Dependencies resolved.
> Nothing to do.
> Complete!
> [jpeach@sk1 tests]$ which python3-config
> /usr/bin/which: no python3-config in (/home/jpeach/bin:/home/jpeach/go/bin:/home/jpeach/bin:/home/jpeach/go/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin)
> [jpeach@sk1 tests]$ pipenv install
> Creating a virtualenv for this project...
> Pipfile: /home/jpeach/src/trafficserver/tests/Pipfile
> Using /usr/bin/python3 (3.11.3) to create virtualenv...
> ⠸ Creating virtual environment...created virtual environment CPython3.11.3.final.0-64 in 156ms
> creator Venv(dest=/home/jpeach/.local/share/virtualenvs/tests-U2EyhSHq, clear=False, no_vcs_ignore=False, global=False, describe=CPython3Posix)
> seeder FromAppData(extra_search_dir=/usr/share/python-wheels,download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/jpeach/.local/share/virtualenv)
> added seed packages: pip==22.3.1, setuptools==65.5.1, wheel==0.38.4
> activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
>
> ✔ Successfully created virtual environment!
> Virtualenv location: /home/jpeach/.local/share/virtualenvs/tests-U2EyhSHq
> Installing dependencies from Pipfile.lock (5beb05)...
> To activate this project's virtualenv, run pipenv shell.
> Alternatively, run a command inside the virtualenv with pipenv run.
> ```
Is it possible you installed the packages locally with python3-devel before this experiment with python3-devel removed? If so, then the subsequent pipenv install will work, even with python3-devel removed, because those dependencies will have been built already from the previous run. As I demonstrated above, a clean system without python3-devel will fail installing the autest package on an initial run (likely because of at least the psutil package). You can reproduce the experiment yourself with the docker steps I outlined above.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@trafficserver.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] bneradt commented on pull request #9910: Add back python3-devel check for autests
Posted by "bneradt (via GitHub)" <gi...@apache.org>.
bneradt commented on PR #9910:
URL: https://github.com/apache/trafficserver/pull/9910#issuecomment-1607952554
[approve ci autest]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@trafficserver.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org