You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@avro.apache.org by "Fedor Telnov (Jira)" <ji...@apache.org> on 2023/06/13 14:28:00 UTC
[jira] [Updated] (AVRO-3780) [Rust] Bug: decimal logical type usage through Fixed
[ https://issues.apache.org/jira/browse/AVRO-3780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Fedor Telnov updated AVRO-3780:
-------------------------------
Description:
Currently, fixed type almost could not be used to represent decimal logical type.
What am I trying to do is to parse such a schema:
```
{
"type": "record",
"name": "recordWithDecimal",
"fields": [
{
"name": "decimal",
"type":
{ "type": "fixed", "name": "nestedFixed", "size": 8, "logicalType": "decimal", "precision": 4 }
}
],
}
```
It must be working if we strictly follow the avro specs. It does not work on master - you can see it through the test I supplied in the attached PR.
Interestingly, such a thing works with bytes, so following schema works great:
```
{
"type": "record",
"name": "recordWithDecimal",
"fields": [
{
"name": "decimal",
"type":
{ "type": "bytes", "name": "nestedFixed", "logicalType": "decimal", "precision": 4 }
}
],
}
```
And the only existing workaround is to extract decimal attributes to the field declaration, like so:
```
{
"name": "decimal",
"type":
{ "type": "fixed", "name": "nestedFixed" }
,
"size": 8,
"logicalType": "decimal",
"precision": 4
}
```
Such a workaround should not be treated as an acceptable variant as it does not follow avro specification at all(I think, such a syntax is an extension, introduced in the Rust crate - correct me if I'm wrong).
I haven't investigated the roots yet, so any thoughts are welcome before I do it myself.
was:
Currently, fixed type almost could not be used to represent decimal logical type.
What am I trying to do is to parse such a schema:
```
{
"type": "record",
"name": "recordWithDecimal",
"fields": [
{ "name": "id", "type": "string" }
,
{
"name": "decimal",
"type":
{ "type": "fixed", "name": "nestedFixed", "size": 8, "logicalType": "decimal", "precision": 4 }
}
],
}
```
It must be working if we strictly follow the avro specs. It does not work on master - you can see it through the test I supplied in the attached PR.
Interestingly, such a thing works with bytes, so following schema works great:
```
{
"type": "record",
"name": "recordWithDecimal",
"fields": [
{
"name": "id",
"type": "string"
},
{
"name": "decimal",
"type": {
"type": "bytes",
"name": "nestedFixed",
"logicalType": "decimal",
"precision": 4
}
}
],
}
```
And the only existing workaround is to extract decimal attributes to the field declaration, like so:
```
{
"name": "decimal",
"type": {
"type": "fixed",
"name": "nestedFixed"
},
"size": 8,
"logicalType": "decimal",
"precision": 4
}
```
Such a workaround should not be treated as an acceptable variant as it does not follow avro specification at all(I think, such a syntax is an extension, introduced in the Rust crate - correct me if I'm wrong).
I haven't investigated the roots yet, so any thoughts are welcome before I do it myself.
> [Rust] Bug: decimal logical type usage through Fixed
> ----------------------------------------------------
>
> Key: AVRO-3780
> URL: https://issues.apache.org/jira/browse/AVRO-3780
> Project: Apache Avro
> Issue Type: Bug
> Components: rust
> Reporter: Fedor Telnov
> Priority: Major
>
> Currently, fixed type almost could not be used to represent decimal logical type.
> What am I trying to do is to parse such a schema:
> ```
> {
> "type": "record",
> "name": "recordWithDecimal",
> "fields": [
> {
> "name": "decimal",
> "type":
> { "type": "fixed", "name": "nestedFixed", "size": 8, "logicalType": "decimal", "precision": 4 }
> }
> ],
> }
> ```
> It must be working if we strictly follow the avro specs. It does not work on master - you can see it through the test I supplied in the attached PR.
> Interestingly, such a thing works with bytes, so following schema works great:
> ```
> {
> "type": "record",
> "name": "recordWithDecimal",
> "fields": [
> {
> "name": "decimal",
> "type":
> { "type": "bytes", "name": "nestedFixed", "logicalType": "decimal", "precision": 4 }
> }
> ],
> }
> ```
>
> And the only existing workaround is to extract decimal attributes to the field declaration, like so:
> ```
> {
> "name": "decimal",
> "type":
> { "type": "fixed", "name": "nestedFixed" }
> ,
> "size": 8,
> "logicalType": "decimal",
> "precision": 4
> }
> ```
> Such a workaround should not be treated as an acceptable variant as it does not follow avro specification at all(I think, such a syntax is an extension, introduced in the Rust crate - correct me if I'm wrong).
>
> I haven't investigated the roots yet, so any thoughts are welcome before I do it myself.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)