You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@parquet.apache.org by "iemejia (via GitHub)" <gi...@apache.org> on 2026/06/02 15:30:04 UTC
[PR] Fix specification typos, grammar, inconsistencies, and errors [parquet-format]
iemejia opened a new pull request, #572:
URL: https://github.com/apache/parquet-format/pull/572
## Summary
This PR fixes numerous typos, grammar issues, inconsistencies, and minor errors across the Parquet format specification documents. The changes span 13 files with 4 focused cleanup commits.
## Changes
### Commit 1: Fix specification inconsistencies, typos, and errors
- **BloomFilter.md**: Fix `block_check` pseudocode (`setBit` -> `isSet`); fix struct name to match thrift
- **parquet.thrift**: Fix typos ("to be be", "documention", "not necessary"); remove off-by-one in DataPageHeaderV2 comment
- **README.md**: Fix repetition level value for non-nested columns (1 -> 0); update defunct Twitter CoC links to ASF
- **LogicalTypes.md**: Fix embedded types ordering contradiction; add nanosecond to TIME precision
- **VariantEncoding.md**: Fix BINARY -> BYTE_ARRAY; add decimal endianness note
- **Compression.md**: Fix ZSTD RFC reference (8478 -> 8878)
- **Encryption.md**: Fix double-negative; align GCM invocation limit to NIST
- **Encodings.md**: Remove misleading "always preferred" claim for DELTA_LENGTH_BYTE_ARRAY
### Commit 2: Fix more specification inconsistencies and clarify ambiguous descriptions
- **PageIndex.md, parquet.thrift**: Fix double-quote typo
- **VariantShredding.md**: Fix Python syntax error; replace BINARY with BYTE_ARRAY
- **BloomFilter.md**: Include missing `bloom_filter_length` field
- **Encodings.md**: "bitwidth of each block" -> "each miniblock"
- **LogicalTypes.md**: Align DECIMAL precision/scale wording with thrift
- **Geospatial.md**: Use uppercase edge-interpolation algorithm names to match thrift enum
- **VariantEncoding.md**: Label undocumented reserved bits; fix decimal implied-precision formula
### Commit 3: Fix additional typos, grammar, invalid HTML, and consistency issues (28 fixes)
- **CONTRIBUTING.md**: 7 typos (docuemnt, interopability, libaries, etc.)
- **Encryption.md**: 6 fixes (plural agreement, explictly, smart quotes, double spaces)
- **LogicalTypes.md**: 7 fixes (invalid `<tr colspan=3>`, NaN casing, grammar)
- **parquet.thrift**: 4 fixes (article agreement, terminal periods, BIT_PACKED comment)
- **Encodings.md, Compression.md, PageIndex.md, VariantEncoding.md**: Minor fixes
### Commit 4: Fix additional typos, grammar, hyphenation, and consistency issues (52 fixes)
- **parquet.thrift**: Article agreement, edge interpolation, proper noun capitalization
- **Geospatial.md**: Compound adjectives, comma splices, heading formatting
- **LogicalTypes.md**: Grammar, Oxford commas, "can not" -> "cannot"
- **README.md**: Plural agreement, compound adjective hyphenation, proper nouns
- **BinaryProtocolExtensions.md**: FileMetaData casing, FlatBuffers capitalization
- **Encodings.md, Compression.md, Encryption.md, BloomFilter.md, PageIndex.md, VariantShredding.md, VariantEncoding.md**: Various grammar, punctuation, and consistency fixes
## Validation
- Thrift definition compiles cleanly after all changes
- No semantic/behavioral changes to the format specification
- All fixes are documentation-only (typos, grammar, consistency, correctness of descriptions)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org
Re: [PR] Fix specification typos, grammar, inconsistencies, and errors [parquet-format]
Posted by "iemejia (via GitHub)" <gi...@apache.org>.
iemejia closed pull request #572: Fix specification typos, grammar, inconsistencies, and errors
URL: https://github.com/apache/parquet-format/pull/572
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org
Re: [PR] Fix specification typos, grammar, inconsistencies, and errors [parquet-format]
Posted by "emkornfield (via GitHub)" <gi...@apache.org>.
emkornfield commented on code in PR #572:
URL: https://github.com/apache/parquet-format/pull/572#discussion_r3350749003
##########
LogicalTypes.md:
##########
@@ -219,15 +219,15 @@ scale stores the number of digits of that value that are to the right of the
decimal point, and the precision stores the maximum number of digits supported
in the unscaled value.
-If not specified, the scale is 0. Scale must be zero or a positive integer less
-than or equal to the precision. Precision is required and must be a non-zero positive
Review Comment:
I think it is still good to call out non-zero here even though it is repetive.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org
Re: [PR] Fix specification typos, grammar, inconsistencies, and errors [parquet-format]
Posted by "emkornfield (via GitHub)" <gi...@apache.org>.
emkornfield commented on code in PR #572:
URL: https://github.com/apache/parquet-format/pull/572#discussion_r3350674108
##########
Compression.md:
##########
@@ -48,7 +48,7 @@ No-op codec. Data is left uncompressed.
A codec based on the
[Snappy compression format](https://github.com/google/snappy/blob/master/format_description.txt).
If any ambiguity arises when implementing this format, the implementation
-provided by Google Snappy [library](https://github.com/google/snappy/)
+provided by the [Snappy compression library](https://github.com/google/snappy/)
Review Comment:
not sure Google's credit is being dropped?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org
Re: [PR] Fix specification typos, grammar, inconsistencies, and errors [parquet-format]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #572:
URL: https://github.com/apache/parquet-format/pull/572#issuecomment-4604320065
Thanks @iemejia - it would help me review this if you could break it into several smaller PRs -- as I think different people have different levels of expertise and I think it will be eaiser to get reviews on smaller proposed changes
For example, I am not sure if the `edge` --> `edges` interpolation is technically accurate but I feel bad pinging the geospatial people on such a large PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org
Re: [PR] Fix specification typos, grammar, inconsistencies, and errors [parquet-format]
Posted by "iemejia (via GitHub)" <gi...@apache.org>.
iemejia commented on PR #572:
URL: https://github.com/apache/parquet-format/pull/572#issuecomment-4605877819
Closing in favor of smaller, concept-focused PRs for easier review:
- #573 — parquet.thrift comment fixes
- #574 — Variant format documentation fixes
- #575 — Geospatial.md fixes
- #576 — Core format docs (README, Encodings, Compression, LogicalTypes)
- #577 — BloomFilter.md and PageIndex.md fixes
- #578 — Encryption, Contributing, and BinaryProtocolExtensions fixes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org
Re: [PR] Fix specification typos, grammar, inconsistencies, and errors [parquet-format]
Posted by "emkornfield (via GitHub)" <gi...@apache.org>.
emkornfield commented on code in PR #572:
URL: https://github.com/apache/parquet-format/pull/572#discussion_r3350689299
##########
Encodings.md:
##########
@@ -54,9 +54,10 @@ Supported Types: all
This is the plain encoding that must be supported for types. It is
intended to be the simplest encoding. Values are encoded back to back.
-The plain encoding is used whenever a more efficient encoding can not be used. It
+The plain encoding is used whenever a more efficient encoding cannot be used. It
stores the data in the following format:
- - BOOLEAN: [Bit Packed](#BITPACKED), LSB first
+ - BOOLEAN: bit-packed, LSB first (using the same packing scheme as the
Review Comment:
this looks like it drops BITPACKED hyper link, does it not exist in the doc?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org
Re: [PR] Fix specification typos, grammar, inconsistencies, and errors [parquet-format]
Posted by "emkornfield (via GitHub)" <gi...@apache.org>.
emkornfield commented on code in PR #572:
URL: https://github.com/apache/parquet-format/pull/572#discussion_r3350730952
##########
Geospatial.md:
##########
@@ -47,8 +47,8 @@ in the order of longitude/latitude based on the WGS84 datum.
Non-default CRS values are specified by any string that uniquely identifies a coordinate reference system associated with this type.
To maximize interoperability, suggested (but not limited to) formats for CRS are:
* `<projjson>` - A complete CRS definition embedded directly using the [PROJJSON](https://proj.org/en/stable/specifications/projjson.html) specification. Example for `OGC:CRS83`: `{"$schema": "https://proj.org/schemas/v0.7/projjson.schema.json","type": "GeographicCRS","name": "NAD83 (CRS83)","datum": {"type": "GeodeticReferenceFrame"...`
-* `<authority>:<code>` - where `<authority>` represents some well known authorities, and `code` is the code used by the authority to identify the CRS. Examples are - `OGC:CRS84`, `OGC:CRS83`, `OGC:CRS27`, `EPSG:4326`, `EPSG:3857`, `IGNF:ATI`. See [https://spatialreference.org/](https://spatialreference.org/) for definitions of coordinate reference systems provided by some well known authorities.
-* `srid:<identifier>` - A reference using a [Spatial reference identifier (SRID)](https://en.wikipedia.org/wiki/Spatial_reference_system#Identifier), where <identifier> is the numeric SRID value. For example: `SRID:0`.
+* `<authority>:<code>` - where `<authority>` represents some well-known authorities, and `code` is the code used by the authority to identify the CRS. Examples are - `OGC:CRS84`, `OGC:CRS83`, `OGC:CRS27`, `EPSG:4326`, `EPSG:3857`, `IGNF:ATI`. See [https://spatialreference.org/](https://spatialreference.org/) for definitions of coordinate reference systems provided by some well-known authorities.
+* `srid:<identifier>` - A reference using a [Spatial reference identifier (SRID)](https://en.wikipedia.org/wiki/Spatial_reference_system#Identifier), where <identifier> is the numeric SRID value. For example: `srid:0`.
Review Comment:
not sure here whether this is expected to be upper-cased or lower cased.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org
Re: [PR] Fix specification typos, grammar, inconsistencies, and errors [parquet-format]
Posted by "emkornfield (via GitHub)" <gi...@apache.org>.
emkornfield commented on code in PR #572:
URL: https://github.com/apache/parquet-format/pull/572#discussion_r3350698342
##########
Encodings.md:
##########
@@ -322,8 +323,6 @@ The delta encoding algorithm described above stores a bit width per miniblock an
Supported Types: BYTE_ARRAY
-This encoding is always preferred over PLAIN for byte array columns.
Review Comment:
why was this dropped?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org
Re: [PR] Fix specification typos, grammar, inconsistencies, and errors [parquet-format]
Posted by "emkornfield (via GitHub)" <gi...@apache.org>.
emkornfield commented on code in PR #572:
URL: https://github.com/apache/parquet-format/pull/572#discussion_r3350740409
##########
Geospatial.md:
##########
@@ -58,19 +58,19 @@ by [-90, 90].
## Edge Interpolation Algorithm
-An algorithm for interpolating edges, and is one of the following values:
+An algorithm for interpolating edges. It is one of the following values:
-* `spherical`: edges are interpolated as geodesics on a sphere.
-* `vincenty`: [https://en.wikipedia.org/wiki/Vincenty%27s_formulae](https://en.wikipedia.org/wiki/Vincenty%27s_formulae)
-* `thomas`: Thomas, Paul D. Spheroidal geodesics, reference systems, & local geometry. US Naval Oceanographic Office, 1970.
-* `andoyer`: Thomas, Paul D. Mathematical models for navigation systems. US Naval Oceanographic Office, 1965.
-* `karney`: [Karney, Charles FF. "Algorithms for geodesics." Journal of Geodesy 87 (2013): 43-55](https://link.springer.com/content/pdf/10.1007/s00190-012-0578-z.pdf), and [GeographicLib](https://geographiclib.sourceforge.io/)
+* `SPHERICAL`: edges are interpolated as geodesics on a sphere.
Review Comment:
not clear if capitalization here. it seems like this is a substantive changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org
Re: [PR] Fix specification typos, grammar, inconsistencies, and errors [parquet-format]
Posted by "emkornfield (via GitHub)" <gi...@apache.org>.
emkornfield commented on code in PR #572:
URL: https://github.com/apache/parquet-format/pull/572#discussion_r3350704604
##########
Encryption.md:
##########
@@ -136,9 +136,9 @@ one IV is ever repeated, then the implementation may be vulnerable"*. *"Complian
requirement is crucial to the security of GCM"*.
The bulk of modules in a Parquet file are page headers and data pages. Therefore, one encryption
-key shall not not be used for more than 2^31 (~2 billion) pages. In Parquet files encrypted with
-multiple keys (footer and column keys), the constraint on the number of invocations is applied
-to each key separately.
+key shall not be used for more than 2^32 total module encryptions, as per the NIST specification.
Review Comment:
this seems like a substantive change?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org