You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by GitBox <gi...@apache.org> on 2020/04/30 07:40:04 UTC
[GitHub] [camel-kafka-connector] orpiske opened a new issue #178: Automate bootstrap for non-containerized deployment?
orpiske opened a new issue #178:
URL: https://github.com/apache/camel-kafka-connector/issues/178
Looking at the [AWS-to-JMS](https://github.com/orpiske/camel-kafka-connector-examples/tree/master/aws-s3-to-jms) example it seems to me that the bootstrap steps are going to be similar most of the time.
1. Download the connector package
2. (Optional) Download the dependencies
3. Create the configuration file
4. Put the files on the Kafka Connect runtime install directory.
I know this is handled automatically that for containerized deployments such as Strimzi, so I was wondering if the project should have a facility for regular deployments.
What do you think? IMHO, it could make even the documentation and example creation simpler, as the project can abstract a few repetitive things and reduce the entry barrier.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] lburgazzoli commented on issue #178: Automate bootstrap for non-containerized deployment?
Posted by GitBox <gi...@apache.org>.
lburgazzoli commented on issue #178:
URL: https://github.com/apache/camel-kafka-connector/issues/178#issuecomment-622930661
looks looks we may need a start.spring.io alike solution but for camel
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] lburgazzoli commented on issue #178: Automate bootstrap for non-containerized deployment?
Posted by GitBox <gi...@apache.org>.
lburgazzoli commented on issue #178:
URL: https://github.com/apache/camel-kafka-connector/issues/178#issuecomment-623270108
>
> @lburgazzoli well actually this would be something different since it does not generate a starting project for you but rather help you to easily run one. Have I missed something?
I think I misinterpreted the subject reading the issue on my mobile as points 1,2,3 looks very similar to what you do with start.spring.io or similar tools.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] orpiske edited a comment on issue #178: Automate bootstrap for non-containerized deployment?
Posted by GitBox <gi...@apache.org>.
orpiske edited a comment on issue #178:
URL: https://github.com/apache/camel-kafka-connector/issues/178#issuecomment-622924221
> It sounds a good idea to me! In which form would you plan to add that?
Thanks.
I was thinking about a CLI utility, in a module within the project. The usage would look like this:
```camel-kafka-connector bootstrap --connector sjms2 --target-dir /path/to/kafka/connect```
It would have 2 additional arguments:
```
--download-host: to allow choosing the host from where to download the files
--override-property: to allow the user to override a setting
```
We already have the list of configurations in the connector config files ([example](https://github.com/apache/camel-kafka-connector/blob/master/connectors/camel-activemq-kafka-connector/src/main/java/org/apache/camel/kafkaconnector/activemq/CamelActivemqSinkConnectorConfig.java)).
It's not clear to me yet how to handle generating the configuration. I think there are 2 options:
1. We could simply generate a configuration file with all the settings.
2. We could generate a configuration file w/ only the required settings.
Option 1 is significantly easier, although the configuration file would be much larger than our examples. Option 2 is more complicated and we would probably need to come up with a mechanism to highlight what is required and what is not.
IMHO, I think we could stick w/ option 1.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] valdar commented on issue #178: Automate bootstrap for non-containerized deployment?
Posted by GitBox <gi...@apache.org>.
valdar commented on issue #178:
URL: https://github.com/apache/camel-kafka-connector/issues/178#issuecomment-622957560
@orpiske For the configuration part actually I was planning to add an autogenerated example configuration listing only the required options (the camel catalog should provide that information). We can potentially convey that information using the Kafka connect config priority field (i.e. only the properties with priority = HIGH are the mandatory ones).
Probably we would need to specify the version as well? Or directly some GAV?
It is a nice idea, it sounds a decent amount of work is needed for a facility to basically run examples and standalone (that nowadays I am not sure how much would be used, as you said containers platforms have these facilities built in somehow.
@lburgazzoli well actually this would be something different since it does not generate a starting project for you but rather help you to easily run one. Have I missed something?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] orpiske commented on issue #178: Automate bootstrap for non-containerized deployment?
Posted by GitBox <gi...@apache.org>.
orpiske commented on issue #178:
URL: https://github.com/apache/camel-kafka-connector/issues/178#issuecomment-623308132
> @orpiske For the configuration part actually I was planning to add an autogenerated example configuration listing only the required options (the camel catalog should provide that information). We can potentially convey that information using the Kafka connect config priority field (i.e. only the properties with priority = HIGH are the mandatory ones).
I think that's a good idea. I think reusing the priority would make it possible to use it for the bootstrapping part as well.
>
> Probably we would need to specify the version as well? Or directly some GAV?
I think we need that too. I think we could use the GAV. IIRC, camel-k CLI already uses the GAV, so overall usage throughout the projects would be consistent.
>
> It is a nice idea, it sounds a decent amount of work is needed for a facility to basically run examples and standalone (that nowadays I am not sure how much would be used, as you said containers platforms have these facilities built in somehow.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] valdar edited a comment on issue #178: Automate bootstrap for non-containerized deployment?
Posted by GitBox <gi...@apache.org>.
valdar edited a comment on issue #178:
URL: https://github.com/apache/camel-kafka-connector/issues/178#issuecomment-622590485
It sounds a good idea to me! In which form would you plan to add that?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] valdar commented on issue #178: Automate bootstrap for non-containerized deployment?
Posted by GitBox <gi...@apache.org>.
valdar commented on issue #178:
URL: https://github.com/apache/camel-kafka-connector/issues/178#issuecomment-622590485
In which form would you plan to ship that?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] valdar edited a comment on issue #178: Automate bootstrap for non-containerized deployment?
Posted by GitBox <gi...@apache.org>.
valdar edited a comment on issue #178:
URL: https://github.com/apache/camel-kafka-connector/issues/178#issuecomment-622590485
In which form would you plan to add that?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] orpiske commented on issue #178: Automate bootstrap for non-containerized deployment?
Posted by GitBox <gi...@apache.org>.
orpiske commented on issue #178:
URL: https://github.com/apache/camel-kafka-connector/issues/178#issuecomment-622924221
> It sounds a good idea to me! In which form would you plan to add that?
Thanks.
I was thinking about a CLI utility, in a module within the project. The usage would look like this:
```camel-kafka-connector bootstrap --connector sjms2 --target-dir /path/to/kafka/connect```
It would have 2 additional arguments:
```
--download-host: to allow choosing the host from where to download the files
--override-property: to allow the user to override a setting
```
We already have the list of configurations in the connector config files ([example](https://github.com/apache/camel-kafka-connector/blob/master/connectors/camel-activemq-kafka-connector/src/main/java/org/apache/camel/kafkaconnector/activemq/CamelActivemqSinkConnectorConfig.java)).
It's not clear to me yet, how to handle generating the configuration. I think there are 2 options:
1. We could simply generate a configuration file with all the settings.
2. We could generate a configuration file w/ only the required settings.
Option 1 is significantly easier, although the configuration file would be much larger than our examples. Option 2 is more complicated and we would probably need to come up with a mechanism to highlight what is required and what is not.
IMHO, I think we could stick w/ option 1.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org