You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@parquet.apache.org by "makarandhinge (via GitHub)" <gi...@apache.org> on 2026/09/19 18:42:42 UTC
[PR] GH-3751: Fix recursive protobuf collection handling [parquet-java]
makarandhinge opened a new pull request, #3802:
URL: https://github.com/apache/parquet-java/pull/3802
### Rationale for this change
Recursive protobuf messages can reach the configured maximum recursion depth when represented as repeated fields or map values.
Previously, this could produce a mismatch between the Parquet schema and the protobuf writer, resulting in a `ClassCastException` when writing recursive repeated fields. Recursive map values could also fail during readback.
### What changes are included in this PR?
- Preserve the `LIST` structure when a recursive repeated protobuf message reaches the maximum recursion depth, storing the element as binary protobuf data.
- Preserve the `MAP` structure when a recursive map value reaches the maximum recursion depth, storing the value as binary protobuf data.
- Update `ProtoWriteSupport` to select writers based on the actual Parquet element/value schema.
- Update `ProtoMessageConverter` to deserialize recursion-truncated binary protobuf messages during readback.
- Add regression tests covering recursive repeated messages and recursive map values using a real `ProtoParquetWriter`.
- Update schema tests and expected Parquet schema resources for the new recursion-boundary representation.
### Are these changes tested?
Yes.
Added end-to-end regression tests for:
- Recursive `Trees.WideTree.children` repeated messages.
- Recursive `google.protobuf.Value` / `Struct.fields` map values.
The full `parquet-protobuf` test suite passes:
```text
Tests run: 111, Failures: 0, Errors: 0, Skipped: 0
BUILD SUCCESS
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org
Re: [PR] GH-3751: Fix recursive protobuf collection handling [parquet-java]
Posted by "makarandhinge (via GitHub)" <gi...@apache.org>.
makarandhinge closed pull request #3802: GH-3751: Fix recursive protobuf collection handling
URL: https://github.com/apache/parquet-java/pull/3802
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@parquet.apache.org
For additional commands, e-mail: issues-help@parquet.apache.org