You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@arrow.apache.org by ap...@apache.org on 2026/10/19 12:24:43 UTC
(arrow) branch main updated: GH-52024: [Python] Fix the codec list in show_info() (#51753)
This is an automated email from the ASF dual-hosted git repository.
pitrou pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/arrow.git
The following commit(s) were added to refs/heads/main by this push:
new 61f24ce27b0 GH-52024: [Python] Fix the codec list in show_info() (#51753)
61f24ce27b0 is described below
commit 61f24ce27b0597258938566665d353ce54bccf60
Author: nishad shabbir <ni...@gmail.com>
AuthorDate: Mon Oct 19 17:54:36 2026 +0530
GH-52024: [Python] Fix the codec list in show_info() (#51753)
### Rationale for this change
`pa.show_info()` prints its Compression Codecs section from a hardcoded list:
```python
codecs = ["brotli", "bz2", "gzip", "lz4_frame", "lz4", "snappy", "zstd"]
```
That list has drifted from `_ensure_compression()` in `io.pxi`, which is what
actually decides which codec names work. Two problems follow:
- `lz4` and `lz4_frame` both map to `CCompressionType_LZ4_FRAME`, so the same
codec is reported twice under two names.
- `lz4_raw` maps to `CCompressionType_LZ4` and is a separate codec, but it is
not listed at all, even though `pa.Codec.is_available('lz4_raw')` is True.
So seven lines are printed for six codecs, and the codec that is missing is a
real one.
Before:
```
Compression Codecs:
brotli : Enabled
bz2 : Enabled
gzip : Enabled
lz4_frame : Enabled
lz4 : Enabled
snappy : Enabled
zstd : Enabled
```
After:
```
Compression Codecs:
brotli : Enabled
bz2 : Enabled
gzip : Enabled
lz4 : Enabled
lz4_raw : Enabled
snappy : Enabled
zstd : Enabled
```
### What changes are included in this PR?
One list in `python/pyarrow/__init__.py`: drop the duplicate `lz4_frame` and add
the missing `lz4_raw`. `lz4` is kept as the name for the frame codec since that
is the one the docstrings lead with (`'lz4' (or 'lz4_frame')`).
On deriving this programmatically rather than hardcoding it — agreed, that is
the better fix. The canonical mapping lives in `_ensure_compression()` in
`io.pxi`, and neither `Codec` nor the C++ `compression.h` currently exposes the
set of supported codecs, so it needs a new accessor on the Cython side for
`show_info()` to read. I am happy to do that here, with the caveat that I cannot
compile the Cython extension in my environment and would be relying on CI to
verify it. If you would prefer, I can keep this change minimal and open a
follow-up issue for the programmatic version. Let me know which you would rather
have.
### Are these changes tested?
Not by a new test. `show_info()` has no existing test coverage and its output is
a diagnostic print, so I did not want to pin the exact text without knowing
whether you would want that — happy to add one if you do.
Verified by applying the same change to an installed pyarrow and reading the
output back, and by walking every name `_ensure_compression()` accepts to
confirm which of them map onto the same C++ enum value:
```
bz2 -> BZ2, gzip -> GZIP, brotli -> BROTLI,
lz4 -> LZ4_FRAME, lz4_frame -> LZ4_FRAME, # same codec, two names
lz4_raw -> LZ4, # distinct, was not listed
snappy -> SNAPPY, zstd -> ZSTD
```
### Are there any user-facing changes?
Only the output of `pa.show_info()`, which now lists each supported codec once
and includes `lz4_raw`. No API change.
### Was AI used for this PR?
In accordance to the [AI generation guidelines](https://arrow.apache.org/docs/dev/developers/overview.html#ai-generated-code), please disclose below whether and how AI was used in this PR.
**PR code and description written by:**
- [x] Human
- [ ] AI
**Reviewed before submission by:**
- [x] Human
- [ ] AI
- [ ] Not reviewed
* GitHub Issue: #52024
Authored-by: nishad shabbir <ni...@gmail.com>
Signed-off-by: Antoine Pitrou <an...@python.org>
---
python/pyarrow/__init__.py | 5 ++++-
python/pyarrow/tests/test_misc.py | 20 ++++++++++++++++++++
2 files changed, 24 insertions(+), 1 deletion(-)
diff --git a/python/pyarrow/__init__.py b/python/pyarrow/__init__.py
index f2a874f56fa..c024cdc69a7 100644
--- a/python/pyarrow/__init__.py
+++ b/python/pyarrow/__init__.py
@@ -156,7 +156,10 @@ def show_info():
print(f" {fs: <20}: {status: <8}")
print("\nCompression Codecs:")
- codecs = ["brotli", "bz2", "gzip", "lz4_frame", "lz4", "snappy", "zstd"]
+ # "lz4" is an alias for "lz4_frame", so the list used to name the same
+ # codec twice and leave out lz4_raw. See _ensure_compression().
+ codecs = ["brotli", "bz2", "gzip", "lz4_frame", "lz4_raw", "snappy",
+ "zstd"]
for codec in codecs:
status = "Enabled" if Codec.is_available(codec) else "-"
print(f" {codec: <20}: {status: <8}")
diff --git a/python/pyarrow/tests/test_misc.py b/python/pyarrow/tests/test_misc.py
index 856873f441b..c0fa484e35e 100644
--- a/python/pyarrow/tests/test_misc.py
+++ b/python/pyarrow/tests/test_misc.py
@@ -123,6 +123,26 @@ def test_runtime_info():
subprocess.check_call([sys.executable, "-c", code], env=env)
+def test_show_info(capsys):
+ pa.show_info()
+ out = capsys.readouterr().out
+
+ # Which codecs and modules are enabled depends on how PyArrow was built,
+ # so only the labels are checked, not their status.
+ assert "Compression Codecs:" in out
+ codecs = [line.split(":")[0].strip()
+ for line in out.split("Compression Codecs:")[1].splitlines()
+ if ":" in line]
+ # "lz4" is an alias for "lz4_frame", so naming both listed one codec
+ # twice and left out lz4_raw.
+ assert codecs == ["brotli", "bz2", "gzip", "lz4_frame", "lz4_raw",
+ "snappy", "zstd"]
+
+ for section in ("Platform:", "Memory:", "Optional modules:",
+ "Filesystems:"):
+ assert section in out
+
+
@pytest.mark.processes
def test_import_at_shutdown():
# GH-38626: importing PyArrow at interpreter shutdown would crash