You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2022/12/05 19:58:24 UTC
[GitHub] [iceberg] Fokko opened a new pull request, #6362: Python: Fix PyArrow import
Fokko opened a new pull request, #6362:
URL: https://github.com/apache/iceberg/pull/6362
Tested this in a fresh docker container:
```
➜ python git:(fd-fix-pyarrow-import) docker run -v `pwd`:/vo/ -t -i python:3.9 bash
root@1252c09f932c:/vo# cd /vo/
root@1252c09f932c:/vo# pip3 install -e ".[s3fs]"
Obtaining file:///vo
Installing build dependencies ... done
Checking if build backend supports build_editable ... done
Getting requirements to build editable ... done
Preparing editable metadata (pyproject.toml) ... done
Collecting mmhash3==3.0.1
Downloading mmhash3-3.0.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (38 kB)
Collecting pyyaml==6.0.0
Downloading PyYAML-6.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (731 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 731.1/731.1 KB 11.0 MB/s eta 0:00:00
Collecting rich==12.6.0
Downloading rich-12.6.0-py3-none-any.whl (237 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 237.5/237.5 KB 9.7 MB/s eta 0:00:00
Requirement already satisfied: requests==2.28.1 in /usr/local/lib/python3.9/site-packages (from pyiceberg==0.2.0) (2.28.1)
Collecting pydantic==1.10.2
Downloading pydantic-1.10.2-py3-none-any.whl (154 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 154.6/154.6 KB 11.9 MB/s eta 0:00:00
Collecting zstandard==0.19.0
Downloading zstandard-0.19.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (2.3 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.3/2.3 MB 18.3 MB/s eta 0:00:00
Collecting fsspec==2022.10.0
Downloading fsspec-2022.10.0-py3-none-any.whl (138 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 138.8/138.8 KB 8.7 MB/s eta 0:00:00
Collecting click==8.1.3
Downloading click-8.1.3-py3-none-any.whl (96 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 96.6/96.6 KB 13.8 MB/s eta 0:00:00
Collecting s3fs==2022.10.0
Downloading s3fs-2022.10.0-py3-none-any.whl (27 kB)
Collecting typing-extensions>=4.1.0
Downloading typing_extensions-4.4.0-py3-none-any.whl (26 kB)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.9/site-packages (from requests==2.28.1->pyiceberg==0.2.0) (3.4)
Requirement already satisfied: charset-normalizer<3,>=2 in /usr/local/lib/python3.9/site-packages (from requests==2.28.1->pyiceberg==0.2.0) (2.1.1)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.9/site-packages (from requests==2.28.1->pyiceberg==0.2.0) (1.26.13)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.9/site-packages (from requests==2.28.1->pyiceberg==0.2.0) (2022.9.24)
Collecting pygments<3.0.0,>=2.6.0
Downloading Pygments-2.13.0-py3-none-any.whl (1.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/1.1 MB 18.6 MB/s eta 0:00:00
Collecting commonmark<0.10.0,>=0.9.0
Downloading commonmark-0.9.1-py2.py3-none-any.whl (51 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 51.1/51.1 KB 11.3 MB/s eta 0:00:00
Collecting aiohttp!=4.0.0a0,!=4.0.0a1
Downloading aiohttp-3.8.3-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (1.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.0/1.0 MB 18.1 MB/s eta 0:00:00
Collecting aiobotocore~=2.4.0
Downloading aiobotocore-2.4.1-py3-none-any.whl (66 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 66.8/66.8 KB 9.0 MB/s eta 0:00:00
Collecting botocore<1.27.60,>=1.27.59
Downloading botocore-1.27.59-py3-none-any.whl (9.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 9.1/9.1 MB 9.3 MB/s eta 0:00:00
Collecting wrapt>=1.10.10
Downloading wrapt-1.14.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (77 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 77.9/77.9 KB 6.4 MB/s eta 0:00:00
Collecting aioitertools>=0.5.1
Downloading aioitertools-0.11.0-py3-none-any.whl (23 kB)
Collecting aiosignal>=1.1.2
Downloading aiosignal-1.3.1-py3-none-any.whl (7.6 kB)
Collecting frozenlist>=1.1.1
Downloading frozenlist-1.3.3-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (159 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 159.2/159.2 KB 9.0 MB/s eta 0:00:00
Collecting async-timeout<5.0,>=4.0.0a3
Downloading async_timeout-4.0.2-py3-none-any.whl (5.8 kB)
Collecting yarl<2.0,>=1.0
Downloading yarl-1.8.2-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (261 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 261.1/261.1 KB 15.8 MB/s eta 0:00:00
Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.9/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->s3fs==2022.10.0->pyiceberg==0.2.0) (22.1.0)
Collecting multidict<7.0,>=4.5
Downloading multidict-6.0.3-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (116 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 116.3/116.3 KB 16.2 MB/s eta 0:00:00
Collecting jmespath<2.0.0,>=0.7.1
Downloading jmespath-1.0.1-py3-none-any.whl (20 kB)
Collecting python-dateutil<3.0.0,>=2.1
Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 247.7/247.7 KB 14.6 MB/s eta 0:00:00
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.9/site-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.27.60,>=1.27.59->aiobotocore~=2.4.0->s3fs==2022.10.0->pyiceberg==0.2.0) (1.16.0)
Building wheels for collected packages: pyiceberg
Building editable for pyiceberg (pyproject.toml) ... done
Created wheel for pyiceberg: filename=pyiceberg-0.2.0-py3-none-any.whl size=6934 sha256=f8420924c2112a81e69f7d39a0d20a7b6e0d99b40f7c4e33533a9a7561238481
Stored in directory: /tmp/pip-ephem-wheel-cache-ico4xse9/wheels/c1/54/64/3aa1db5712f8c9873a8d54f10706a5c226c66d849f8163ffb9
Successfully built pyiceberg
Installing collected packages: mmhash3, commonmark, zstandard, wrapt, typing-extensions, pyyaml, python-dateutil, pygments, multidict, jmespath, fsspec, frozenlist, click, async-timeout, yarl, rich, pydantic,
botocore, aiosignal, aioitertools, pyiceberg, aiohttp, aiobotocore, s3fs
Successfully installed aiobotocore-2.4.1 aiohttp-3.8.3 aioitertools-0.11.0 aiosignal-1.3.1 async-timeout-4.0.2 botocore-1.27.59 click-8.1.3 commonmark-0.9.1 frozenlist-1.3.3 fsspec-2022.10.0 jmespath-1.0.1
mmhash3-3.0.1 multidict-6.0.3 pydantic-1.10.2 pygments-2.13.0 pyiceberg-0.2.0 python-dateutil-2.8.2 pyyaml-6.0 rich-12.6.0 s3fs-2022.10.0 typing-extensions-4.4.0 wrapt-1.14.1 yarl-1.8.2 zstandard-0.19.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead:
https://pip.pypa.io/warnings/venv
WARNING: You are using pip version 22.0.4; however, version 22.3.1 is available.
You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.
root@1252c09f932c:/vo# cat ~/.pyiceberg.yaml
catalog:
default:
uri: https://api.tabulardata.io/ws/
credential: abc
py-io-impl: pyiceberg.io.fsspec.FsspecFileIO
root@1252c09f932c:/vo# pyiceberg list nyc
nyc.taxis
root@1252c09f932c:/vo# pyiceberg files nyc.taxis
Snapshots: default.nyc.taxis
└── Snapshot 8048355899640248710, schema 0:
s3://tabular-wh-us-west-2-dev/8bcb0838-50fc-472d-9ddb-8feb89ef5f1e/d0489abc-b7be-4731-aab5-47e994092fbf/metadata/snap-8048355899640248710-1-a5c8ea2d-aa1f-48e8-89f4-1fa69db8c742.avro
└── Manifest: s3://tabular-wh-us-west-2-dev/8bcb0838-50fc-472d-9ddb-8feb89ef5f1e/d0489abc-b7be-4731-aab5-47e994092fbf/metadata/a5c8ea2d-aa1f-48e8-89f4-1fa69db8c742-m0.avro
└── Datafile: s3://tabular-wh-us-west-2-dev/8bcb0838-50fc-472d-9ddb-8feb89ef5f1e/d0489abc-b7be-4731-aab5-47e994092fbf/data/00003-4-7b4e5cc3-686c-4c58-8c1f-0a0a3f1ff6e0-00001.parquet
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on pull request #6362: Python: Fix PyArrow import
Posted by GitBox <gi...@apache.org>.
rdblue commented on PR #6362:
URL: https://github.com/apache/iceberg/pull/6362#issuecomment-1339971821
@Fokko, does this mean we need a new RC?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue merged pull request #6362: Python: Fix PyArrow import
Posted by GitBox <gi...@apache.org>.
rdblue merged PR #6362:
URL: https://github.com/apache/iceberg/pull/6362
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org