[GitHub] [camel-k] lburgazzoli edited a comment on issue #1980: Add support for multiple data types and schemas in Kamelets

[GitBox <gi...@apache.org>]

lburgazzoli edited a comment on issue #1980:
URL: https://github.com/apache/camel-k/issues/1980#issuecomment-771480486


   > > I like this idea and would add the possibility for:
   > > 
   > > * references to external schemas (I.e. in schema registry)
   > > * Schemas attached to the message in transit (I.e in header)
   > 
   > Yeah, these are concerns we need to address now as well. The `schema` prop is set to JSONSchema currently, but we need to address other kinds of schemas, including schemas located elsewhere.
   > 
   > I think it's a good time to deprecate `spec` -> `types` and provide something like `spec` -> `dataTypes` just to build a different tree and provide oob migration from old Kamelets.
   > 
   
   Maybe we can use "schemes" instead. 
   
   > For the "schema in header", I think it's a good idea for sources. We can make sure the operator passes the location of the schema in a configuration property and, in case the schema is inline, it also mount it as a file in the pod, so that the header can always be an URL. The Kamelet runtime may also bind that property into a header. The destination (or an intermediate step) can the use that URL to do stuff.
   > 
   
   I think we can improve data formats in general, as example:
   
   1. they can automatically compute the schema at runtime if not provided and store the result in an header
   2. they can use a provided scheme to validate that the marshalled/unmarshalled data conforms with the given schema
   
   We can then define a specific schema like:
   
   ```yaml
   avro:
     media-type: application/avro
     schema:
       # the avro schema inline|reference
     data-format:
       # optional, if not provided use the scheme id 
       id: "avro"
       properties:
         class-name: org.apache.camel.xxx.MyClass
         compute-schema: true|false
         ...
     dependencies:
       - camel-avro
       - mvn:org.acme/my-artifact/1.0.0
   ```
   
   
   > Wdyt @lburgazzoli ?
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org