You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@parquet.apache.org by Steve Loughran <st...@cloudera.com> on 2026/03/30 13:09:57 UTC
is it (VARIANT) or (VARIANT(1))?
The normative specification of variants, parquet-format/VariantEncoding.md,
uses examples
optional group shredded_variant_name (VARIANT) {
required binary metadata;
optional binary value;
optional int64 typed_value;
}
But copy that and you get an error
java.lang.IllegalArgumentException: Expecting only spec version for variant
annotation args: []
at org.apache.parquet.Preconditions.checkArgument(Preconditions.java:70)
at
org.apache.parquet.schema.LogicalTypeAnnotation$LogicalTypeToken$3.fromString(LogicalTypeAnnotation.java:64)
at
org.apache.parquet.schema.MessageTypeParser.addGroupType(MessageTypeParser.java:139)
at
org.apache.parquet.schema.MessageTypeParser.addType(MessageTypeParser.java:109)
at
org.apache.parquet.schema.MessageTypeParser.addGroupTypeFields(MessageTypeParser.java:99)
at
org.apache.parquet.schema.MessageTypeParser.parse(MessageTypeParser.java:92)
at
org.apache.parquet.schema.MessageTypeParser.parseMessageType(MessageTypeParser.java:82)
It has to be (VARIANT(1)), as is done in the limited number of places a
variant is declared in the code.
Is this an omission in the specification?
If it is, I'll file an issue/pr to fix.
Re: is it (VARIANT) or (VARIANT(1))?
Posted by Steve Loughran <st...@cloudera.com>.
PR for the fix https://github.com/apache/parquet-format/pull/562
On Tue, 31 Mar 2026 at 13:52, Steve Loughran <st...@cloudera.com> wrote:
>
>
> On Mon, 30 Mar 2026 at 15:47, Aihua Xu <ai...@gmail.com> wrote:
>
>> Yeah. It should be VARIANT(1). We have some places using VARIANT
>> incorrectly - we may not catch them when we finalized the spec.
>
>
> thanks. That's what I suspected.
>
>>
>>
Re: is it (VARIANT) or (VARIANT(1))?
Posted by Steve Loughran <st...@cloudera.com>.
On Mon, 30 Mar 2026 at 15:47, Aihua Xu <ai...@gmail.com> wrote:
> Yeah. It should be VARIANT(1). We have some places using VARIANT
> incorrectly - we may not catch them when we finalized the spec.
thanks. That's what I suspected.
>
>
Re: is it (VARIANT) or (VARIANT(1))?
Posted by Aihua Xu <ai...@gmail.com>.
Yeah. It should be VARIANT(1). We have some places using VARIANT incorrectly - we may not catch them when we finalized the spec.
> On Mar 30, 2026, at 6:17 AM, Andrew Lamb <an...@gmail.com> wrote:
>
> I believe it needs needs to be VARIANT(1) , the (1) being the
> "specification_version" [1]
>
> We recently fixed an example in LogicalTypes.md[2] to follow the .thrift
>
> Fixing the examples in VariantEncoding.md[3] I think would be a good change
>
> Andrew
>
> [1]:
> https://github.com/apache/parquet-format/blob/285b6fd1335da2e8bafe8570f0c00b55c2c41748/src/main/thrift/parquet.thrift#L412
> [2]: https://github.com/apache/parquet-format/pull/555
> [3]: https://github.com/apache/parquet-format/blob/master/VariantEncoding.md
>
>> On Mon, Mar 30, 2026 at 9:10 AM Steve Loughran <st...@cloudera.com> wrote:
>>
>> The normative specification of variants, parquet-format/VariantEncoding.md,
>> uses examples
>>
>> optional group shredded_variant_name (VARIANT) {
>> required binary metadata;
>> optional binary value;
>> optional int64 typed_value;
>> }
>>
>> But copy that and you get an error
>>
>> java.lang.IllegalArgumentException: Expecting only spec version for variant
>> annotation args: []
>> at org.apache.parquet.Preconditions.checkArgument(Preconditions.java:70)
>> at
>>
>> org.apache.parquet.schema.LogicalTypeAnnotation$LogicalTypeToken$3.fromString(LogicalTypeAnnotation.java:64)
>> at
>>
>> org.apache.parquet.schema.MessageTypeParser.addGroupType(MessageTypeParser.java:139)
>> at
>>
>> org.apache.parquet.schema.MessageTypeParser.addType(MessageTypeParser.java:109)
>> at
>>
>> org.apache.parquet.schema.MessageTypeParser.addGroupTypeFields(MessageTypeParser.java:99)
>> at
>>
>> org.apache.parquet.schema.MessageTypeParser.parse(MessageTypeParser.java:92)
>> at
>>
>> org.apache.parquet.schema.MessageTypeParser.parseMessageType(MessageTypeParser.java:82)
>>
>> It has to be (VARIANT(1)), as is done in the limited number of places a
>> variant is declared in the code.
>>
>> Is this an omission in the specification?
>>
>> If it is, I'll file an issue/pr to fix.
>>
Re: is it (VARIANT) or (VARIANT(1))?
Posted by Andrew Lamb <an...@gmail.com>.
I believe it needs needs to be VARIANT(1) , the (1) being the
"specification_version" [1]
We recently fixed an example in LogicalTypes.md[2] to follow the .thrift
Fixing the examples in VariantEncoding.md[3] I think would be a good change
Andrew
[1]:
https://github.com/apache/parquet-format/blob/285b6fd1335da2e8bafe8570f0c00b55c2c41748/src/main/thrift/parquet.thrift#L412
[2]: https://github.com/apache/parquet-format/pull/555
[3]: https://github.com/apache/parquet-format/blob/master/VariantEncoding.md
On Mon, Mar 30, 2026 at 9:10 AM Steve Loughran <st...@cloudera.com> wrote:
> The normative specification of variants, parquet-format/VariantEncoding.md,
> uses examples
>
> optional group shredded_variant_name (VARIANT) {
> required binary metadata;
> optional binary value;
> optional int64 typed_value;
> }
>
> But copy that and you get an error
>
> java.lang.IllegalArgumentException: Expecting only spec version for variant
> annotation args: []
> at org.apache.parquet.Preconditions.checkArgument(Preconditions.java:70)
> at
>
> org.apache.parquet.schema.LogicalTypeAnnotation$LogicalTypeToken$3.fromString(LogicalTypeAnnotation.java:64)
> at
>
> org.apache.parquet.schema.MessageTypeParser.addGroupType(MessageTypeParser.java:139)
> at
>
> org.apache.parquet.schema.MessageTypeParser.addType(MessageTypeParser.java:109)
> at
>
> org.apache.parquet.schema.MessageTypeParser.addGroupTypeFields(MessageTypeParser.java:99)
> at
>
> org.apache.parquet.schema.MessageTypeParser.parse(MessageTypeParser.java:92)
> at
>
> org.apache.parquet.schema.MessageTypeParser.parseMessageType(MessageTypeParser.java:82)
>
> It has to be (VARIANT(1)), as is done in the limited number of places a
> variant is declared in the code.
>
> Is this an omission in the specification?
>
> If it is, I'll file an issue/pr to fix.
>