You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2022/07/02 17:19:00 UTC

[jira] [Work logged] (AVRO-3528) Optionally support strict LogicalType parsing

     [ https://issues.apache.org/jira/browse/AVRO-3528?focusedWorklogId=787340&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-787340 ]

ASF GitHub Bot logged work on AVRO-3528:
----------------------------------------

                Author: ASF GitHub Bot
            Created on: 02/Jul/22 17:18
            Start Date: 02/Jul/22 17:18
    Worklog Time Spent: 10m 
      Work Description: m8719-github opened a new pull request, #1752:
URL: https://github.com/apache/avro/pull/1752

   ### Jira
   
   - [x] https://issues.apache.org/jira/browse/AVRO-3528
   
   ### Tests
   
   - [x] Adds the following unit tests:
   `TestLogicalType.testInvalidLogicalTypeIncompatibleProps`
   `TestLogicalType.testStrictLogicalTypesParsing`
   
   ### Commits
   
   - [x] My commits all reference Jira issues in their subject lines. In addition, my commits follow the guidelines from "[How to write a good git commit message](https://chris.beams.io/posts/git-commit/)":
     1. Subject is separated from body by a blank line
     1. Subject is limited to 50 characters (not including Jira issue reference)
     1. Subject does not end with a period
     1. Subject uses the imperative mood ("add", not "adding")
     1. Body wraps at 72 characters
     1. Body explains "what" and "why", not "how"
   
   ### Documentation
   
   - [x] In case of new functionality, my PR adds documentation that describes how to use it.
     - All the public functions and the classes in the PR contain Javadoc that explain what it does
   




Issue Time Tracking
-------------------

            Worklog Id:     (was: 787340)
    Remaining Estimate: 0h
            Time Spent: 10m

> Optionally support strict LogicalType parsing
> ---------------------------------------------
>
>                 Key: AVRO-3528
>                 URL: https://issues.apache.org/jira/browse/AVRO-3528
>             Project: Apache Avro
>          Issue Type: Improvement
>          Components: java
>    Affects Versions: 1.11.0
>            Reporter: Andrei Leibovski
>            Priority: Minor
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> My organization uses Avro schemas extensively. We use Confluent schema registry for data governance, enforcing data contracts between various components. We are seeing proliferation of questionable LogicalType structures within our schemas, like the following:
> {code:java}
> {
>   "namespace": "org.apache.avro.example",
>   "type": "record",
>   "name": "BadLogical",
>   "fields": [
>    {
>      "name": "f0",
>      "type":{
>        "type": "string",
>        "java-class": "java.math.BigDecimal",
>        "logicalType": "decimal",
>        "precision": 9,
>        "scale": 2
>      }
>    }
>   ]
> } {code}
> There are two issues in the above structure:
>  # string is not allowed to back the decimal LogicalType
>  # java-class property and some others, are incompatible for any LogicalType
> Currently Avro allows such structures to pass validation and has no option to disallow them. Since Confluent schema registry delegates all avro schema validation to Avro, these structures are allowed to be registered.
> *Proposition:*
> Implement an option to switch avro Schema.Parser to LogicalType strict mode where such structures will be detected and disallowed.
> The change is fairly trivial, most of the plumbing is already there. I have implemented the required functionality and covered it with with tests, but I would like to solicit some feedback on this proposal before submitting my PR.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)