[I] [Avro] Generate schemas from Pinot logical data types [pinot]

["xiangfu0 (via GitHub)" <gi...@apache.org>]

xiangfu0 opened a new issue, #19071:
URL: https://github.com/apache/pinot/issues/19071

   ### Problem
   
   `AvroUtils.getAvroSchemaFromPinotSchema()` currently switches on `fieldSpec.getDataType().getStoredType()`, so the generated Avro schema describes Pinot's physical storage representation rather than its logical type. In particular:
   
   - `BOOLEAN` is emitted as Avro `int`.
   - `TIMESTAMP` is emitted as Avro `long` without a logical type.
   - `BIG_DECIMAL` falls through as unsupported.
   - UUID requires a one-off special case in #18870.
   
   ### Expected direction
   
   Switch schema generation on the actual Pinot data type and add the coordinated writer/value conversions needed for the resulting Avro types. Cover `BOOLEAN`, `BIG_DECIMAL`, and `TIMESTAMP`, and avoid adding more one-off handling where a shared logical-type mapping/conversion contract can own the behavior.
   
   ### Compatibility and tests
   
   - Preserve compatibility for consumers of previously generated Avro data/schema where required.
   - Verify supported single-value and multi-value combinations.
   - Add Pinot schema to Avro schema assertions for the affected logical types.
   - Add end-to-end write/read round-trip tests using the same data model/conversions as segment-processing writers.
   - Document the chosen Avro logical types and physical representations.
   
   ### Context
   
   Follow-up requested during review of #18870: https://github.com/apache/pinot/pull/18870#discussion_r3641794939


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

Re: [I] [Avro] Generate schemas from Pinot logical data types [pinot]

["xiangfu0 (via GitHub)" <gi...@apache.org>]

xiangfu0 closed issue #19071: [Avro] Generate schemas from Pinot logical data types
URL: https://github.com/apache/pinot/issues/19071


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org