You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by GitBox <gi...@apache.org> on 2022/11/03 16:14:20 UTC
[GitHub] [camel-k] nicolaferraro opened a new issue, #1980: Add support for multiple data types and schemas in Kamelets
nicolaferraro opened a new issue, #1980:
URL: https://github.com/apache/camel-k/issues/1980
Seeking for help on improving the Kamelet model before going full on the Kamelet catalog effort.
Currently the model expects that one declares the default input/output of a Kamelet in the `spec` -> `types` -> `in` / `out` field, like:
```yaml
# ...
types:
out:
mediaType: application/json
schema:
#...
```
The same for Kamelets that consume an input, but the property is named `in`.
The meaning of those types is simply stated:
- A Kamelet produces output with the format specified in `out`
- A Kamelet consumes an input with the format specified in `in`
That has unfortunately some drawbacks, one of which is that a Kamelet must have a single data type as output (for sources) and/or a single data type for input.
Many implementations of Kamelets that produce JSON data, in fact, have the following route snippet in the flow part so far:
```yaml
# ...
steps:
- marshal:
json: {}
```
So e.g., if we go full with the Kamelet catalog and add support for them in camel-kafka-connector, I expect soon to have a `salesforce-source-json` and a `salesforce-source-avro` to overcome this limitation. But it's not ideal.
I think we should allow a Kamelet to have a default input/output format, without forcing users to use that one: they may have choices.
I was thinking to something like this:
```yaml
# ...
types:
out:
mediaType: application/json
schema:
#...
camel:
dataFormat: json-jackson
```
The `dataFormat` option tells the operator to automatically add `camel:jackson` and the marshalling step when the Kamelet is used in a KameletBinding.
For `in`, this translates into:
```yaml
# ...
types:
in:
mediaType: application/json
schema:
#...
camel:
dataFormat: json-jackson
className: org.apache.camel.xxx.MyClass
```
This should add the unmarshalling to a specific (optional) class.
In case we want this behavior to be common for KameletBinding and standard integration, **this should be better implemented at runtime level**.
Now the question is how to deal with the case of multiple input/output data types.
A possibility would be to add another level of description:
```yaml
# ...
types:
out:
default: json
json:
mediaType: application/json
schema:
# the JSON schema
camel:
dataFormat: json-jackson
avro:
mediaType: application/avro
schema:
# the Avro schema
camel:
dataFormat: avro
className: org.apache.camel.xxx.MyClass
```
That would break a bit the current schema, but it will provide more options in the future.
Having the possibility to choose, a user can specify the **format** option in a KameletBinding (that we're going to reserve, like we did for **id**), to select an input/output format that is different from the default (maybe including `none`, to obtain the original data in advanced use cases).
In case this should work also in the standard integration, we may use the following syntax:
```
from("kamelet:salesforce-source?format=avro").to("....")
```
From the operator side, the required libraries for Avro will be added, but the runtime should enhance the route with a loader/customizer.
Wdyt @lburgazzoli, @astefanutti , @davsclaus, @squakez
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-k] github-actions[bot] commented on issue #1980: Add support for multiple data types and schemas in Kamelets
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #1980:
URL: https://github.com/apache/camel-k/issues/1980#issuecomment-1104573645
This issue has been automatically marked as stale due to 90 days of inactivity.
It will be closed if no further activity occurs within 15 days.
If you think that’s incorrect or the issue should never stale, please simply write any comment.
Thanks for your contributions!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-k] github-actions[bot] closed issue #1980: Add support for multiple data types and schemas in Kamelets
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed issue #1980: Add support for multiple data types and schemas in Kamelets
URL: https://github.com/apache/camel-k/issues/1980
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-k] squakez closed issue #1980: Add support for multiple data types and schemas in Kamelets
Posted by "squakez (via GitHub)" <gi...@apache.org>.
squakez closed issue #1980: Add support for multiple data types and schemas in Kamelets
URL: https://github.com/apache/camel-k/issues/1980
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org