You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/05/03 22:47:21 UTC
[GitHub] [pulsar] devinbost opened a new issue, #15417: Unable to run Pulsar Python client on M1 Mac
devinbost opened a new issue, #15417:
URL: https://github.com/apache/pulsar/issues/15417
I can't find a way to get the Pulsar Python client working on an M1 Mac.
Here are my steps / attempts:
```
conda create -n pulsardemo
conda activate pulsardemo
pip install pulsar-client
# Fails to install. Gives:
ERROR: Could not find a version that satisfies the requirement pulsar-client (from versions: none)
ERROR: No matching distribution found for pulsar-client
# Attempted solution #1:
curl https://bootstrap.pypa.io/get-pip.py | python
pip install pulsar-client
pip install --upgrade setuptools
# No luck yet
python --version
pip install pulsar-client==2.2.1
conda search python
python --version
conda install python=3.9.12
pip install pulsar-client
# Installs successfully!
python
>>> import pulsar
[1] 89737 illegal hardware instruction python
# Death...
# Next attempt:
file $(which python3)
python -c "import platform;print(platform.machine())"
conda config --env --set subdir osx-64
pip install pulsar-client
python
>>> import pulsar
[1] 89737 illegal hardware instruction python
# Death... Next attempt:
pip uninstall pulsar-client
softwareupdate --install-rosetta
conda create -n pulsarrosetta
conda activate pulsarrosetta
pip install pulsar-client
conda install pip
pip install pulsar-client
# No luck.
conda install python=3.9.12
pip install pulsar-client
python
>>> import pulsar
[1] 89737 illegal hardware instruction python
# Death again...
# Next, I repeated the steps from a new terminal that I started using rosetta. Same results.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] tisonkun closed issue #15417: Unable to run Pulsar Python client on M1 Mac, including when building from source on master
Posted by GitBox <gi...@apache.org>.
tisonkun closed issue #15417: Unable to run Pulsar Python client on M1 Mac, including when building from source on master
URL: https://github.com/apache/pulsar/issues/15417
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] merlimat commented on issue #15417: Unable to run Pulsar Python client on M1 Mac, including when building from source on master
Posted by GitBox <gi...@apache.org>.
merlimat commented on issue #15417:
URL: https://github.com/apache/pulsar/issues/15417#issuecomment-1117714534
> This pip instance must be tied to a different Python binary.
It looks you're installing in Python 3.8 and trying to use it from Python 3.9
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] devinbost commented on issue #15417: Unable to run Pulsar Python client on M1 Mac, including when building from source on master
Posted by GitBox <gi...@apache.org>.
devinbost commented on issue #15417:
URL: https://github.com/apache/pulsar/issues/15417#issuecomment-1117677742
Hi Matteo,
I get the error `not a supported wheel on this platform`. e.g.
```
(pulsardemo) ➜ dist git:(master) pip install --verbose pulsar_client-2.11.0-cp310-cp310-macosx_11_0_universal2.whl
Using pip 21.2.4 from /Users/dbost/opt/anaconda3/envs/pulsardemo/lib/python3.10/site-packages/pip (python 3.10)
ERROR: pulsar_client-2.11.0-cp310-cp310-macosx_11_0_universal2.whl is not a supported wheel on this platform.
```
I've attached the output of `pip debug --verbose` (since it had a lot of output):
[pip_debug.log](https://github.com/apache/pulsar/files/8626198/pip_debug.log)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] devinbost commented on issue #15417: Unable to run Pulsar Python client on M1 Mac, including when building from source on master
Posted by GitBox <gi...@apache.org>.
devinbost commented on issue #15417:
URL: https://github.com/apache/pulsar/issues/15417#issuecomment-1117664645
@merlimat I heard this is something you've worked on.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] devinbost commented on issue #15417: Unable to run Pulsar Python client on M1 Mac, including when building from source on master
Posted by GitBox <gi...@apache.org>.
devinbost commented on issue #15417:
URL: https://github.com/apache/pulsar/issues/15417#issuecomment-1117722300
I tracked down the path to pip that was installed by brew.
When I tried installing into that pip, I got the `No matching distribution` error I was seeing when I tried installing via anaconda for Python 2.10:
```
➜ pulsar-dumps /opt/homebrew/Cellar/python@3.9/3.9.12/bin/pip3 install python-client
DEPRECATION: Configuring installation scheme with distutils config files is deprecated and will no longer work in the near future. If you are using a Homebrew or Linuxbrew Python, please see discussion at https://github.com/Homebrew/homebrew-core/issues/76621
ERROR: Could not find a version that satisfies the requirement python-client (from versions: none)
ERROR: No matching distribution found for python-client
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on issue #15417: Unable to run Pulsar Python client on M1 Mac, including when building from source on master
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #15417:
URL: https://github.com/apache/pulsar/issues/15417#issuecomment-1146497961
The issue had no activity for 30 days, mark with Stale label.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] devinbost commented on issue #15417: Unable to run Pulsar Python client on M1 Mac, including when building from source on master
Posted by GitBox <gi...@apache.org>.
devinbost commented on issue #15417:
URL: https://github.com/apache/pulsar/issues/15417#issuecomment-1117709141
I got the Python version via Anaconda.
Here's what happens when I try doing it on the OS directly. (I must be missing something.)
```
➜ pulsar-dumps python3 -m pip install --upgrade pip
Defaulting to user installation because normal site-packages is not writeable
Collecting pip
Using cached pip-22.0.4-py3-none-any.whl (2.1 MB)
Installing collected packages: pip
WARNING: The scripts pip, pip3 and pip3.8 are installed in '/Users/dbost/Library/Python/3.8/bin' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed pip-22.0.4
➜ pulsar-dumps export PATH=/Users/dbost/Library/Python/3.8/bin:$PATH
➜ pulsar-dumps pip install pulsar-client
Defaulting to user installation because normal site-packages is not writeable
Collecting pulsar-client
Downloading pulsar_client-2.10.0-cp38-cp38-macosx_11_0_universal2.whl (10.3 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.3/10.3 MB 45.4 MB/s eta 0:00:00
Collecting certifi
Downloading certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 149.2/149.2 KB 5.2 MB/s eta 0:00:00
Requirement already satisfied: six in /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages (from pulsar-client) (1.15.0)
Installing collected packages: certifi, pulsar-client
Successfully installed certifi-2021.10.8 pulsar-client-2.10.0
➜ pulsar-dumps python3
Python 3.9.12 (main, Mar 26 2022, 15:44:31)
[Clang 13.1.6 (clang-1316.0.21.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pulsar
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'pulsar'
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] merlimat commented on issue #15417: Unable to run Pulsar Python client on M1 Mac, including when building from source on master
Posted by GitBox <gi...@apache.org>.
merlimat commented on issue #15417:
URL: https://github.com/apache/pulsar/issues/15417#issuecomment-1117687204
I'm trying to see it's possible to target Mac 10.15 in a general way. I got errors in the past though I don't remember exactly what.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] merlimat commented on issue #15417: Unable to run Pulsar Python client on M1 Mac, including when building from source on master
Posted by GitBox <gi...@apache.org>.
merlimat commented on issue #15417:
URL: https://github.com/apache/pulsar/issues/15417#issuecomment-1117681273
It looks like the python you have installed is not compiled with macOS 11.0 support.
```
cp310-cp310-macosx_10_16_universal2
...
```
How did you get that Python version?
Can you try with the Python from `brew`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] devinbost commented on issue #15417: Unable to run Pulsar Python client on M1 Mac
Posted by GitBox <gi...@apache.org>.
devinbost commented on issue #15417:
URL: https://github.com/apache/pulsar/issues/15417#issuecomment-1117663588
I was able to build from source from master locally on my M1 Mac, but when I try to install the pip wheels, I get:
> ERROR: [wheel-file].whl is not a supported wheel on this platform.
(I was running Python 3.9.12)
I tried all of the wheel files produced by the build. They are:
- pulsar_client-2.11.0-cp310-cp310-macosx_11_0_universal2.whl
- pulsar_client-2.11.0-cp39-cp39-macosx_11_0_universal2.whl
- pulsar_client-2.11.0-cp37-cp37m-macosx_11_0_x86_64.whl
- pulsar_client-2.8.0-cp37-cp37m-linux_x86_64.whl
- pulsar_client-2.11.0-cp38-cp38-macosx_11_0_universal2.whl
I also tried upgrading Python to 3.10.4 and installing `pulsar_client-2.11.0-cp310-cp310-macosx_11_0_universal2.whl` but that also gave me the same error.
Reproduction steps. Build from source on M1 Mac:
```
sudo xcodebuild -license
brew install boost
brew install protobuf
brew install openssl
brew link --force openssl
export PATH="/opt/homebrew/opt/openssl@3/bin:$PATH"
echo 'export PATH="/opt/homebrew/opt/openssl@3/bin:$PATH"' >> ~/.zshrc
export OPENSSL_INCLUDE_DIR=/opt/homebrew/Cellar/openssl@3/3.0.3/include
export OPENSSL_ROOT_DIR=/opt/homebrew/Cellar/openssl@3/3.0.3
brew install libcurl
# Downloaded latest stable release of curllib
cd /Users/dbost/proj/binaries/curl-7.83.0
./configure --prefix=/usr/local/curl --with-openssl=/opt/homebrew/Cellar/openssl@3/3.0.3
make
sudo make install
which curl
# Downloaded latest stable gtest release
cd ..
mv /Users/dbost/Downloads/googletest-release-1.11.0.tar.gz .
cp /Users/dbost/Downloads/googletest-release-1.11.0.tar.gz /Users/dbost/proj/binaries/
cd /Users/dbost/proj/binaries/
tar -zxvf /Users/dbost/Downloads/googletest-release-1.11.0.tar.gz
cd googletest-release-1.11.0
mkdir build
cd ~/proj/apache-pulsar/pulsar
git pull
cd pulsar-client-cpp
brew install protobuf boost boost-python3 log4cxx jsoncpp
# Installed cmake from GUI. Then:
export PATH=/Applications/CMake.app/Contents/bin:$PATH
cd /Users/dbost/proj/binaries/googletest-release-1.11.0
sudo cmake .
sudo make
sudo cp *.a /usr/lib # Didn't work. Next attempt:
sudo cp ./lib/*.a /usr/lib # Didn't work. Next attempt:
sudo cp ./lib/*.a /usr/local/lib
cd ~/proj/apache-pulsar
cd pulsar
cd pulsar-client-cpp
sudo ln -s /Applications/CMake.app/Contents/bin/cmake /usr/local/bin/cmake
brew install gcc
# Failed when I tried to use gcc-11 and g++-11 due to unknown parameters
#export CC=/opt/homebrew/bin/gcc-11
#export CXX=/opt/homebrew/bin/g++-11
export CC=/usr/bin/gcc
export CXX=/usr/bin/g++
export PULSAR_PATH=/Users/dbost/proj/apache-pulsar/pulsar
rm CMakeCache.txt
cmake . -DPYTHON_INCLUDE_DIR=$(python3 -c "from distutils.sysconfig import get_python_inc; print(get_python_inc())") -DPYTHON_LIBRARY=$(python3 -c "import distutils.sysconfig as sysconfig; print(sysconfig.get_config_var('LIBDIR'))")
# Failed, so I tried excluding the tests:
rm CMakeCache.txt
cmake . -DPYTHON_INCLUDE_DIR=$(python3 -c "from distutils.sysconfig import get_python_inc; print(get_python_inc())") -DPYTHON_LIBRARY=$(python3 -c "import distutils.sysconfig as sysconfig; print(sysconfig.get_config_var('LIBDIR'))") -DBUILD_TESTS=OFF
make
# Success!
cd python
vi build-mac-wheels.sh
./build-mac-wheels.sh
# Success!
cd dist
conda create -n pulsarexample
conda install wheel
conda activate pulsarexample
pip install (any of the wheel files)
# They all fail
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] merlimat commented on issue #15417: Unable to run Pulsar Python client on M1 Mac, including when building from source on master
Posted by GitBox <gi...@apache.org>.
merlimat commented on issue #15417:
URL: https://github.com/apache/pulsar/issues/15417#issuecomment-1117669045
```
./build-mac-wheels.sh
# Success!
cd dist
conda create -n pulsarexample
conda install wheel
conda activate pulsarexample
pip install (any of the wheel files)
# They all fail
```
How are these failing? What error are you getting?
Can you try with the official released wheels? https://pypi.org/project/pulsar-client/2.10.0/#files
Can you also report the `pip debug --verbose` (this would include all the tags supported by your python env).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] devinbost commented on issue #15417: Unable to run Pulsar Python client on M1 Mac, including when building from source on master
Posted by GitBox <gi...@apache.org>.
devinbost commented on issue #15417:
URL: https://github.com/apache/pulsar/issues/15417#issuecomment-1117724832
I tracked down the path to the pip that was installed by brew, and when I installed into that pip, it worked!
```
➜ pulsar-dumps /opt/homebrew/Cellar/python@3.9/3.9.12/bin/pip3 install pulsar-client
DEPRECATION: Configuring installation scheme with distutils config files is deprecated and will no longer work in the near future. If you are using a Homebrew or Linuxbrew Python, please see discussion at https://github.com/Homebrew/homebrew-core/issues/76621
Collecting pulsar-client
Downloading pulsar_client-2.10.0-cp39-cp39-macosx_11_0_universal2.whl (10.3 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.3/10.3 MB 74.0 MB/s eta 0:00:00
Collecting certifi
Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting six
Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Installing collected packages: certifi, six, pulsar-client
DEPRECATION: Configuring installation scheme with distutils config files is deprecated and will no longer work in the near future. If you are using a Homebrew or Linuxbrew Python, please see discussion at https://github.com/Homebrew/homebrew-core/issues/76621
DEPRECATION: Configuring installation scheme with distutils config files is deprecated and will no longer work in the near future. If you are using a Homebrew or Linuxbrew Python, please see discussion at https://github.com/Homebrew/homebrew-core/issues/76621
DEPRECATION: Configuring installation scheme with distutils config files is deprecated and will no longer work in the near future. If you are using a Homebrew or Linuxbrew Python, please see discussion at https://github.com/Homebrew/homebrew-core/issues/76621
DEPRECATION: Configuring installation scheme with distutils config files is deprecated and will no longer work in the near future. If you are using a Homebrew or Linuxbrew Python, please see discussion at https://github.com/Homebrew/homebrew-core/issues/76621
Successfully installed certifi-2021.10.8 pulsar-client-2.10.0 six-1.16.0
➜ pulsar-dumps python3
Python 3.9.12 (main, Mar 26 2022, 15:44:31)
[Clang 13.1.6 (clang-1316.0.21.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pulsar
# No error this time!
```
Thanks for helping me think through this.
I guess the next question is why it wasn't working from Anaconda.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] devinbost commented on issue #15417: Unable to run Pulsar Python client on M1 Mac, including when building from source on master
Posted by GitBox <gi...@apache.org>.
devinbost commented on issue #15417:
URL: https://github.com/apache/pulsar/issues/15417#issuecomment-1117715072
Even more strange...
```
➜ pulsar-dumps which pip
/Users/dbost/Library/Python/3.8/bin/pip
```
Here's how I installed pip:
```
➜ pulsar-dumps python3 -m ensurepip --upgrade
Defaulting to user installation because normal site-packages is not writeable
Looking in links: /var/folders/dp/4wpkd0x11fjf749z311_7h340000gp/T/tmpg3g7sib9
Requirement already up-to-date: setuptools in /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages (49.2.1)
Requirement already up-to-date: pip in /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages (20.2.3)
➜ pulsar-dumps pip
zsh: command not found: pip
➜ pulsar-dumps python get-pip.py
zsh: command not found: python
➜ pulsar-dumps python -m pip install --upgrade pip
zsh: command not found: python
➜ pulsar-dumps python3 -m pip install --upgrade pip
Defaulting to user installation because normal site-packages is not writeable
Collecting pip
Using cached pip-22.0.4-py3-none-any.whl (2.1 MB)
Installing collected packages: pip
WARNING: The scripts pip, pip3 and pip3.8 are installed in '/Users/dbost/Library/Python/3.8/bin' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed pip-22.0.4
➜ pulsar-dumps export PATH=/Users/dbost/Library/Python/3.8/bin:$PATH
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] devinbost commented on issue #15417: Unable to run Pulsar Python client on M1 Mac, including when building from source on master
Posted by GitBox <gi...@apache.org>.
devinbost commented on issue #15417:
URL: https://github.com/apache/pulsar/issues/15417#issuecomment-1117680038
The official released wheels are what I tried first. (I provided reproduction steps that used those wheels in my first entry in this thread.)
I was able to install one of the official wheels by using python 3.9.12, but when I tried importing it, I got:
`illegal hardware instruction python`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] tisonkun commented on issue #15417: Unable to run Pulsar Python client on M1 Mac, including when building from source on master
Posted by GitBox <gi...@apache.org>.
tisonkun commented on issue #15417:
URL: https://github.com/apache/pulsar/issues/15417#issuecomment-1309810838
It seems the original issue has been resolved. Closed.
The development of the Python client has been permanently moved to https://github.com/apache/pulsar-client-python. Please open an issue there if any.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org