You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "tritzman (via GitHub)" <gi...@apache.org> on 2024/04/18 02:20:46 UTC
[I] [Python][Parquet] Using decryption_properties parameter on pq.ParquetDataset raises exception [arrow]
tritzman opened a new issue, #41273:
URL: https://github.com/apache/arrow/issues/41273
### Describe the bug, including details regarding any error messages, version, and platform.
pyarrow 15.0.2
Changing the read method for example at python/examples/parquet_encryption/sample_vault_kms_client.py
from this:
`result = pq.ParquetFile(path,decryption_properties=file_decryption_properties)`
to this:
`result = pq.ParquetDataset(path,decryption_properties=file_decryption_properties)`
results in this exception:
`__init__() got an unexpected keyword argument 'decryption_properties'
Tracking it down, `Parquet.ParquetDataset` supports the parameter `decryption_properties`. The `decryption_properties` are passed through `__init__` where it’s collected in the `read_options`. The `read_options` are passed to `ds.ParquetFileFormat` (on line 1337 of core.py). The `decryption_properties` are not accepted by `ds.ParqetFileFormat`, which raises the exception.
I was looking at https://github.com/apache/arrow/issues/29238, thinking the changes in v14.0.0 might have affected the way decryption properties are used with datasets (vs files). It's unclear to me how one uses decrpytion_properties with a dataset.
### Component(s)
Parquet, Python
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Python][Parquet] Using decryption_properties parameter on pq.ParquetDataset raises exception [arrow]
Posted by "tritzman (via GitHub)" <gi...@apache.org>.
tritzman closed issue #41273: [Python][Parquet] Using decryption_properties parameter on pq.ParquetDataset raises exception
URL: https://github.com/apache/arrow/issues/41273
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Python][Parquet] Using decryption_properties parameter on pq.ParquetDataset raises exception [arrow]
Posted by "tritzman (via GitHub)" <gi...@apache.org>.
tritzman commented on issue #41273:
URL: https://github.com/apache/arrow/issues/41273#issuecomment-2408387002
Seems to be resolved in pyarrow 17.0.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Python][Parquet] Using decryption_properties parameter on pq.ParquetDataset raises exception [arrow]
Posted by "heyuqi1970 (via GitHub)" <gi...@apache.org>.
heyuqi1970 commented on issue #41273:
URL: https://github.com/apache/arrow/issues/41273#issuecomment-2228869001
pyarrow 16.0.0
macos 11.7.10 (20G1427)
python 3.9.7
result = pq.ParquetDataset(path,decryption_properties=file_decryption_properties)
return same exception
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org