You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@camel.apache.org by GitBox <gi...@apache.org> on 2022/01/05 22:48:44 UTC

[GitHub] [camel-kamelets] ReggieCarey opened a new issue #673: Kamelet for ElasticSearch Sink provides no mechanism to set indexId or ce-indexId

ReggieCarey opened a new issue #673:
URL: https://github.com/apache/camel-kamelets/issues/673

Greetings,

I've got a Kamelet that sources a KafkaTopic and sinks to ElasticSearch.

The cloud event payload is JSON.

The payload does contain a unique identifier that I want to use as indexId, I cannot figure out how to use "insert-field-action" or "insert-header-action" to inject indexId/ce-indexId with a unique value from the payload (or for that matter ce-attributes). Attempts to use "extract-field-action" might be of use but there does not appear to be any way to actually use what was extracted. (I was hoping I could extract a field and then use the contents to insert a header or insert a field).

Frankly the Kamelet documentation is useless. It provides no data other than boilerplate as if automatically generated. No explanation of fields, their use, or meaningful examples. See the next three code blocks as exemplars:

```
steps:
- ref:
kind: Kamelet
apiVersion: camel.apache.org/v1alpha1
name: insert-field-action
properties:
field: The Field
value: The Value
```

```
steps:
- ref:
kind: Kamelet
apiVersion: camel.apache.org/v1alpha1
name: extract-field-action
properties:
field: The Field
```

```
steps:
- ref:
kind: Kamelet
apiVersion: camel.apache.org/v1alpha1
name: insert-header-action
properties:
name: headername
value: The Value
```

Do I have to roll my own Kamelet to do something so obviously simple?

I must be missing something.

Thanks,
Reggie

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [camel-kamelets] lburgazzoli commented on issue #673: Kamelet for ElasticSearch Sink provides no mechanism to set indexId or ce-indexId

Posted by GitBox <gi...@apache.org>.

lburgazzoli commented on issue #673:
URL: https://github.com/apache/camel-kamelets/issues/673#issuecomment-1006781799


   > 
   > Listing language specific dependencies in this scenario is a bad code smell. What value is there in telling your audience about your various Java dependencies for your code? It may be of value to people writing more in-depth camel integrations or raw custom kamelets but in general your implementation details should not be a concern to people consuming the solution at the K8s integration tier. Does any of that dependency info in the docs result in something actionable by camel-k users?
   > 
   
   For a user POV what matter are the properties the kamelet exposes, like for any camel endpoint. 
   
   Then as for each artifact, a kamelet CR come with some additional metadata which in this case are the dependencies and routes. Such metadata is needed by the machinery, like the camel-k operator, tooling and other stuffs to work but no one expects users to look at them.
   
   Hope this clarifies at least this aspect.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [camel-kamelets] ReggieCarey commented on issue #673: Kamelet for ElasticSearch Sink provides no mechanism to set indexId or ce-indexId

Posted by GitBox <gi...@apache.org>.

ReggieCarey commented on issue #673:
URL: https://github.com/apache/camel-kamelets/issues/673#issuecomment-1007527212

@oscerd Thanks for all of the effort, I think this mod adds a huge amount of capability.

But lots of questions and some suggestions:

I'm curious about what it implies to use "Simple" and not Constant for the value parameter binding. This would be very handy in the docs. At a high level, I think it allows for strings with embedded expansions "${some_variable}" but that's just an assumption on my part. Could you add into the documentation what variables we have access to in this expansion and their types (Map, Array, Object, etc.) , e.g. `header` as in "${header[ce-source]}", "${header.ce-source}".

I see a potential problem down the road with using a single fixed header (CamelKameletsExtractFieldName). What happens when I need to extract more than one thing to be placed in the header?

Could you do something like this? Note the "field" as well as alternate use of "headerOutput". If we know the payload is JSON, then "field" needs to be interpreted to identify any entry in that json. Ideally this would use a common syntax such as that used by [JQ](https://stedolan.github.io/jq/)

B.T.W. what happens if the thing requested in "field" is not a simple type (string, boolean, number)?

```
- route:
from:
uri: "kamelet:kafka-source"
...
steps:
- to:
uri: "kamelet:extract-field-action"
parameters:
field: "endUser[3].familyName"
headerOutput: "FAMILY_NAME"
- to:
uri: "kamelet:extract-field-action"
parameters:
field: "endUser[3].givenName"
headerOutput: "GIVEN_NAME"
- to:
uri: "kamelet:insert-header-action"
parameters:
name: "ce-indexId"
value: "${header.FAMILY_NAME}_${header.GIVEN_NAME}"
or
value: "${header[FAMILY_NAME]}_${header[GIVEN_NAME]}"
- to:
uri: "kamelet:elasticsearch-index-sink"
...
```

If my understanding of "Simple" is correct, then in a **KameletBinding** one could establish the headerOutputs in steps A and B and then combine the headers in a subsequent step C as per ```value: "${header.FAMILY_NAME}_${header.GIVEN_NAME}"```. Am I understanding the intent properly?

Does the Camel DSL or some subset extend to KameletBindings or is it limited to Kamelets? How much room for configuration do people have if all they want to do is blend existing Kamelets into a solution?

SPECULATION:
I see a need for a generic CloudEvent payload processor Action at some point in the future that gives the ability to read and manipulate any payload not just ones that are JSON encoded. Maybe it is limited to payloads defined with a json schema or protocol buffer or some such to make accessing the contents more predictable. It might involve introducing a block of user defined code (groovy or python script block?) added to the kameletbinding yaml that executes in some sort of sandbox.

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [camel-kamelets] oscerd commented on issue #673: Kamelet for ElasticSearch Sink provides no mechanism to set indexId or ce-indexId

Posted by GitBox <gi...@apache.org>.

oscerd commented on issue #673:
URL: https://github.com/apache/camel-kamelets/issues/673#issuecomment-1006946738


   It will be available in 0.7.0


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [camel-kamelets] oscerd commented on issue #673: Kamelet for ElasticSearch Sink provides no mechanism to set indexId or ce-indexId

Posted by GitBox <gi...@apache.org>.

oscerd commented on issue #673:
URL: https://github.com/apache/camel-kamelets/issues/673#issuecomment-1006330101


   The documentation needs to be improved. Contributions are always welcome. This is an open source project, anyone could help. What is unacceptable is reading something like the generated docs is useless, this is really unpleasant to read for the contributors working on this. I'll try to provide an example tomorrow or next week for this particular case.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [camel-kamelets] djencks commented on issue #673: Kamelet for ElasticSearch Sink provides no mechanism to set indexId or ce-indexId

Posted by GitBox <gi...@apache.org>.

djencks commented on issue #673:
URL: https://github.com/apache/camel-kamelets/issues/673#issuecomment-1007208538


   Yes, but the version is wrong, it should be `https://github.com/apache/camel-kamelets/blob/0.5.x/{name}.kamelet.yaml` <https://github.com/apache/camel-kamelets/blob/0.5.x/%7Bname%7D.kamelet.yaml%60>
   
   It’s too late here to make a PR, but if you don’t get to it I’ll do it first thing tomorrow.
   The version is wrong in 0.6.x also.
   
   David Jencks
   
   > On Jan 6, 2022, at 11:47 PM, Andrea Cosentino ***@***.***> wrote:
   > 
   > 
   > On 0.5.x the kamelets where still in the main folder, so links should be ok.
   > 
   > —
   > Reply to this email directly, view it on GitHub <https://github.com/apache/camel-kamelets/issues/673#issuecomment-1007204896>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAELDXXQBIS37L64EQBLRMTUU2LAJANCNFSM5LLAMB2A>.
   > You are receiving this because you commented.
   > 
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [camel-kamelets] ReggieCarey commented on issue #673: Kamelet for ElasticSearch Sink provides no mechanism to set indexId or ce-indexId

Posted by GitBox <gi...@apache.org>.

ReggieCarey commented on issue #673:
URL: https://github.com/apache/camel-kamelets/issues/673#issuecomment-1006926777


   ### SOLUTION?
   
   I have devised a solution to my basic underlying problem - When we sink to ElasticSearch, instead of hardcoding an indexId as the default if none is present, we remove the "otherwise" clause in the elasticsearch-index-sink Kamelet.
   
   WAS:
   ```
     flow:
       from:
         steps:
         - choice:
             otherwise:
               steps:
               - set-header:
                   name: indexId
                   simple: camel-k-index-es
             when:
             - simple: ${header[indexId]}
               steps:
               - set-header:
                   name: indexId
                   simple: ${header[indexId]}
             - simple: ${header[ce-indexid]}
               steps:
               - set-header:
                   name: indexId
                   simple: ${header[ce-indexid]}
   ```
   
   REMOVE:
   ```
             otherwise:
               steps:
               - set-header:
                   name: indexId
                   simple: camel-k-index-es
   ```
   
   RESULTING IN:
   
   ```
     flow:
       from:
         steps:
         - choice:
             when:
             - simple: ${header[indexId]}
               steps:
               - set-header:
                   name: indexId
                   simple: ${header[indexId]}
             - simple: ${header[ce-indexid]}
               steps:
               - set-header:
                   name: indexId
                   simple: ${header[ce-indexid]}
   ```
   
   This allows a blank indexId to be passed to ElasticSearch allowing it to autogenerate an indexId.
   
   @oscerd I hope this is a reasonable mod to the Kamelet.
   
   Sincerely,
   
   Reggie


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [camel-kamelets] lburgazzoli edited a comment on issue #673: Kamelet for ElasticSearch Sink provides no mechanism to set indexId or ce-indexId

Posted by GitBox <gi...@apache.org>.

lburgazzoli edited a comment on issue #673:
URL: https://github.com/apache/camel-kamelets/issues/673#issuecomment-1006436054


   >     * AFAICT the user guide doesn't mention actions.  IIUC these have both a source and sink, so you can string them together.  An example in the user guide would be great.
   >
   
   The `kamelets`  documentation certainly need some refactoring and improvements but please bear in mind that `kamelets` are still  under heavy development. We are trying to make the `kamelets` and the related ecosystem a generic `camel` concept rather that a `camel-k` specific thing so there are things that are still missing or poorly documented. 
   
   However if you look at the [kamelet-component](https://camel.apache.org/components/3.14.x/kamelet-component.html) document, I think it is pretty easy to understand what the various actions types are meant to be used as they are pretty much like any other component/eip. 
   
   You are very welcome to improve the documentation to include a paragraph about actions.
    
   >     * The kamelets in the catalog have what appear to be toy sources and sinks.  Is their function for testing?  Mentioning this in the user guide might be helpful.
   > 
   
   It would be nice to report what the various sources and sinks are missing.  
   
   Bear in mind that `kamelets` are opinionated and aimed to hide the complexity of the underlying components/EIPs so  don't expect them to offer the same flexibility as the underlying components, if you need that flexibility then you probably should not look at `kamelets`.
   
   > 
   > I also think the description of extract field action is pretty close to useless.... "Extract a field from the body".... and then what? 
   
   I agree that in general there should be a better description of what an action is doing but about the `then what`, well it depends on what you want to achieve. Like when using the setHeader EIP you know that the EIP sets an header, that's all,  isn't it ?
   
   > Trying to find the source code behind this one, it appears to be in ` - "github:apache.camel-kamelets:camel-kamelets-utils:main-SNAPSHOT"` which I can't locate and which by the "main-SNAPSHOT" makes me wonder if it violates the Apache policy against directing people to unreleased code.
   > 
   
   The code is available in this repository, according to the doc [dependencies](https://camel.apache.org/camel-k/1.7.x/configuration/dependencies.html), the `github:` syntax is about using [jitpack](https://jitpack.io/) as a way to get the latest snapshot.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [camel-kamelets] davsclaus commented on issue #673: Kamelet for ElasticSearch Sink provides no mechanism to set indexId or ce-indexId

Posted by GitBox <gi...@apache.org>.

davsclaus commented on issue #673:
URL: https://github.com/apache/camel-kamelets/issues/673#issuecomment-1007250024


   > OK, so I read https://camel.apache.org/camel-k/next/kamelets/kamelets-user.html and have perhaps some clues and some questions.
   > 
   > * I think it would be helpful to directly link to the user guide from the kamelets catalog index page, something like "consult the user guide for information about how to use these".
   
   Yeah the page already refers to the developer guide. Adding a paragraph prior pointing to the user guide is a good idea. And you are surely welcome to send a PR.
   
   
   > * AFAICT the user guide doesn't mention actions.  IIUC these have both a source and sink, so you can string them together.  An example in the user guide would be great.
   
   Actions was added later, we started with sink/sources. You are welcome to create a ticket so the task is not forgotten.
   
   > * The kamelets in the catalog have what appear to be toy sources and sinks.  Is their function for testing?  Mentioning this in the user guide might be helpful.
   
   David, that is a harsh statement to call it "toys" - Kamelets are in heavy development and they are on 0.x release (not reached a 1.0 GA yet). Sours and Sinks are the most important goals of Kamelets, to make it even easier to connect to/from systems.
   
   Take a moment to read also some of the blogs about Kamelets that we have published, such as
   https://camel.apache.org/blog/2021/11/low-code-camel/
   https://camel.apache.org/blog/2021/02/Kamelet-Catalog/
   
   
   > 
   > I also think the description of extract field action is pretty close to useless.... "Extract a field from the body".... and then what? Trying to find the source code behind this one, it appears to be in ` - "github:apache.camel-kamelets:camel-kamelets-utils:main-SNAPSHOT"` which I can't locate and which by the "main-SNAPSHOT" makes me wonder if it violates the Apache policy against directing people to unreleased code.
   > 
   
   David, this is again a harsh statement about "violation of XXX". As you have learned now there is a reason for this as explained by Luca. 
   
   
   > Personally, when someone says the docs are useless, I tend to interpret it as "I'm so frustrated I can't figure out how to do what I want!!!" and try to respond to that.
   
   Okay but you did not only do that .. you also made acquisitions and harsh remarks about the camel-kamelets project, which made this ticket worse and causing frustrations among the maintainers of this project.
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [camel-kamelets] ReggieCarey commented on issue #673: Kamelet for ElasticSearch Sink provides no mechanism to set indexId or ce-indexId

Posted by GitBox <gi...@apache.org>.

ReggieCarey commented on issue #673:
URL: https://github.com/apache/camel-kamelets/issues/673#issuecomment-1006743329

I'm glad this issue has sparked a lot of conversation. Consider this some documentation feedback.

1) The user community can't help with the documentation if we have no clue as to how the code works or how it was intended to work.
2) The links to the kamelet code in the kameletbinding docs usually end up at a GitHub 404 page.
3) When the links do work, its generally to a yaml file instead of the underlying code that someone could explore to improve the documentation.
4) The kameletbinding documentation has been largely untouched for at least 6 months - as noted its autogenerated.

Camel-K developers: You've done an excellent job with the product. I should point out that you'll get much more constructive feedback and increased usage if you bring the documentation up to minimum viable product level. The user community would love to provide constructive feedback but we are working with what appears to be a lot of barriers to success and its due primarily to not being able to understand how these yaml files (KameletBindings) can be used to get something done. I'll say it again

```
steps:
- ref:
kind: Kamelet
apiVersion: camel.apache.org/v1alpha1
name: insert-field-action
properties:
field: The Field
value: The Value
```

is not adequate documentation especially for a product that boasts releasing version 3.11.5 on 31 Dec 2021. But the above is ALL the documentation there is on this capability. Disheartening as it is to hear, the documentation sucks.

What are appropriate values for "field"? What are legal values for "value"? Is there a way to pull values from the cloud event headers? Is there a syntax you're abiding by to specify where in a json payload the 'value' will be placed? Is there a templating tool to bring some intelligence to "The Field" and "The Value" strings?

Can I pull values from previous steps of the kameletbinding? If so, how?

Is there more to the syntax than what is shown in the KameletBinding exemplars? If so, where is it documented?

What can we do? What functionality is present?

What can we not do? What functionality is missing or broken?

Basically, if the feature is not complete, please help us help you. Tell us what's missing, what's broken, what's planned. Include a link in the documentation for the feature if not to the specific code implementing the feature, then at least to the repo containing the code.

A note on your documentation audience:

Users coming to Kamelets from a Knative/CloudEvent/Kamel Operator perspective interact with your product via resources deployed into a K8s cluster. Listing language specific dependencies in this scenario is a bad code smell. What value is there in telling your audience about your various Java dependencies for your code? It may be of value to people writing more in-depth camel integrations or raw custom kamelets but in general your implementation details should not be a concern to people consuming the solution at the K8s integration tier. Does any of that dependency info in the docs result in something actionable by camel-k users?

I submit that your documentation should focus on how to use all of the nifty things you've built into those Kamelets used in KameletBinding files (the semantics) instead of just what makes a syntactically correct yaml file.

With respect,

Reggie

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [camel-kamelets] djencks commented on issue #673: Kamelet for ElasticSearch Sink provides no mechanism to set indexId or ce-indexId

Posted by GitBox <gi...@apache.org>.

djencks commented on issue #673:
URL: https://github.com/apache/camel-kamelets/issues/673#issuecomment-1006164897


   Well, the per-kamelet docs are completely automatically generated from the yml kamelet descriptor.  I share your frustration.  Writing the latest version of the docs generation I was hoping to understand what could be customized in a kamelet and am still completely mystified.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [camel-kamelets] oscerd commented on issue #673: Kamelet for ElasticSearch Sink provides no mechanism to set indexId or ce-indexId

Posted by GitBox <gi...@apache.org>.

oscerd commented on issue #673:
URL: https://github.com/apache/camel-kamelets/issues/673#issuecomment-1007157005

You weren't asking for this fix, but for something else, using the extract action for setting the indexId from the json routing in the integration. This is not a solution, it's just a workaround.

Thanks for reporting about the links, I'll take care of fixing them. Maybe there were some updates when we changed the way the documentation generation with some PRs from David.

I will always speak up when the work from this community and from the oldest contributors will be underrated and I really wish that members of this PMC do the same, because in the last weeks I just seen attempts to criticize the work done and find faults in what it is done (if someone feel I'm talking about him, yes, I'm talking about you, David).

20 hours ago I couldn't answer about the fix, because your problem was and it's still different.

The 100's hours of time you consumed, it's 500 hours of work done from contributors around Kamelets and camel-k. The fact that you don't know the code or you have troubles finding where the code is or how it is working, it's part of the game. Camel is a really complex ecosystem, documenting everything will always be impossible, given the number of real contributors working regularly and with passion on this.

I'll take comments like yours as offence, because you and many others, consider, ALWAYS, our time for granted, which it's wrong and unpleasant to read.

That say, thanks for your feedback and I'll trying to improve the upstream documentation, in my spare time, as always.

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [camel-kamelets] lburgazzoli commented on issue #673: Kamelet for ElasticSearch Sink provides no mechanism to set indexId or ce-indexId

Posted by GitBox <gi...@apache.org>.

lburgazzoli commented on issue #673:
URL: https://github.com/apache/camel-kamelets/issues/673#issuecomment-1008088423


   
   > 
   > Does the Camel DSL or some subset extend to KameletBindings or is it limited to Kamelets?  How much room for configuration do people have if all they want to do is blend existing Kamelets into a solution?
   > 
   
   No, the goal of KameletBinding (which should be renamed to Binding) is to abstract the underlying engine so we do not expect to add any Camel DSL capability directly to the Binding, if you need such flexibility then you should think about using an Integration where you can use any Camel DSL.
   
   > 
   > I see a need for a generic CloudEvent payload processor Action at some point in the future that gives the ability to read and manipulate any payload not just ones that are JSON encoded.  Maybe it is limited to payloads defined with a json schema or protocol buffer or some such to make accessing the contents more predictable.  It might involve introducing a block of user defined code (groovy or python script block?) added to the kameletbinding yaml that executes in some sort of sandbox.
   > 
   
   Adding scripting capabilities is not an issue through a Kamelet is pretty simple. 
   
   But I want to point you that this has almost nothing to do with CloudEvents as a CloudEventn is nothing more than an envelope that matches what in Camel is a Message (as Andrea pointed out, there is an issue about a better CliudEvent support in Camel). Camel offers already a way to manipulate Messages (of course) by using a scripting language. 
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [camel-kamelets] lburgazzoli commented on issue #673: Kamelet for ElasticSearch Sink provides no mechanism to set indexId or ce-indexId

Posted by GitBox <gi...@apache.org>.

lburgazzoli commented on issue #673:
URL: https://github.com/apache/camel-kamelets/issues/673#issuecomment-1006436054


   >     * AFAICT the user guide doesn't mention actions.  IIUC these have both a source and sink, so you can string them together.  An example in the user guide would be great.
   >
   
   The `kamelets`  documentation certainly need some refactoring and improvements but please bear in mind that `kamelets` are still  under heavy development. We are trying to make the `kamelets` and the related ecosystem a generic `camel` concept rather that a `camel-k` specific thing so there are things that are still missing or poorly documented. 
   
   However if you look at the [kamelet-component](https://camel.apache.org/components/3.14.x/kamelet-component.html) document, I think it is pretty easy to understand what the various actions types are meant to be used as they are pretty much like any other component/eip. 
   
   You are very welcome to improve the documentation to include a paragraph about actions.
    
   >     * The kamelets in the catalog have what appear to be toy sources and sinks.  Is their function for testing?  Mentioning this in the user guide might be helpful.
   > 
   
   It would be nice to report what the various sources and sinks are missing.  
   
   Bear in mind that `kamelets` are opinionated and aimed to hide the complexity of the underlying components/EIPs so  don't expect them to offer the same flexibility as the underlying components, if you need that flexibility then you probably should not look at `kamelets`.
   
   > 
   > I also think the description of extract field action is pretty close to useless.... "Extract a field from the body".... and then what? 
   
   I agree that in general there should be a better description of what an action is doing but about the `then what`, well it depends on what you want to achieve. Like when using the setHeader EIP you know that the the EIP sets an header, that's all,  isn't it ?
   
   > Trying to find the source code behind this one, it appears to be in ` - "github:apache.camel-kamelets:camel-kamelets-utils:main-SNAPSHOT"` which I can't locate and which by the "main-SNAPSHOT" makes me wonder if it violates the Apache policy against directing people to unreleased code.
   > 
   
   The code is available in this repository, according to the doc [dependencies](https://camel.apache.org/camel-k/1.7.x/configuration/dependencies.html), the `github:` syntax is about using [jitpack](https://jitpack.io/) as a way to get the latest snapshot.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [camel-kamelets] djencks commented on issue #673: Kamelet for ElasticSearch Sink provides no mechanism to set indexId or ce-indexId

Posted by GitBox <gi...@apache.org>.

djencks commented on issue #673:
URL: https://github.com/apache/camel-kamelets/issues/673#issuecomment-1006430064


   I found camel-kamelets-utils.... I had my search set wrong in IDEA.  I'll look into what the extract field action is doing more tomorrow.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [camel-kamelets] oscerd commented on issue #673: Kamelet for ElasticSearch Sink provides no mechanism to set indexId or ce-indexId

Posted by GitBox <gi...@apache.org>.

oscerd commented on issue #673:
URL: https://github.com/apache/camel-kamelets/issues/673#issuecomment-1006331035


   David, probably you don't have an idea of how customizing a kamelet because you're only focusing on docs and website. Maybe if you ever try to run a real kamelets you will have an initial idea of what you're talking about.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [camel-kamelets] ReggieCarey commented on issue #673: Kamelet for ElasticSearch Sink provides no mechanism to set indexId or ce-indexId

Posted by GitBox <gi...@apache.org>.

ReggieCarey commented on issue #673:
URL: https://github.com/apache/camel-kamelets/issues/673#issuecomment-1006988572

I have been a huge fan and promoter of Camel-K since I discovered it approximately 1 year ago. It's a very useful tool and can dramatically reduce large system complexity. I'm more than aware of the alpha / pre-alpha quality of some of the components. As such, I've tried to work around the various shortcomings without filing bugs.

@oscerd thank you for making this repair.

When the community says the documentation useless, don't take offense, sometimes the truth hurts. Take it as an opportunity to make things clearer for the community. The documentation around KameletBindings as described in the Kamelet Catalog fails to provide sufficient detail to use the software effectively. I hope I provided some suggestions that clarify what these actions do so that it's not so confusing to others.

The Open Source developers have a dramatically better insight into what's going on under the hood. As consumers of the solution, we often do not have insight into the inner workings of the code. This makes it impossible to provide constructive feedback. Many users of this technology have no Java background and digging through code ends up being a fruitless effort for them. The only recourse is to file a bug/open an issue. I've been working this problem for at least 6 months and have consumed 100's of hours of time and resources on this. Feedback in this issue chain helped me find the code that lead to better understanding of what these actions are doing.

I hope my comments and suggestions lead to updated documentation and code that is better for all who wish to use this open source product.

By the way: If you're not already aware. All of the links in the Kamelet Catalog that point to the Kamelet Source Files are broken and lead to a 404 on GitHub.

On a personal note: I wish you had responded with this latest bit of information about a fix 20 hours ago instead of waiting for me to discover the same or similar solution before telling us that there is already a solution developed 17 days ago.

I was busy trying to find a solution based on combining and working with actions documented in the Kamelet Catalog. Now I know, having read some of the code, that there is likely no solution to the problem by attempting to manipulate the configuration of one or more actions in a kameletbinding.

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [camel-kamelets] oscerd edited a comment on issue #673: Kamelet for ElasticSearch Sink provides no mechanism to set indexId or ce-indexId

Posted by GitBox <gi...@apache.org>.

oscerd edited a comment on issue #673:
URL: https://github.com/apache/camel-kamelets/issues/673#issuecomment-1007157005

You weren't asking for this fix, but for something else, using the extract action for setting the indexId from the json routing in the integration. This is not a solution, it's just a workaround.

Thanks for reporting about the links, I'll take care of fixing them. Maybe there were some updates when we changed the way the documentation generation with some PRs from David.

20 hours ago I couldn't answer about the fix, because your problem was and it's still different.

I'll take comments like yours as offence, because you and many others, consider, ALWAYS, our time for granted, which it's wrong and unpleasant to read.

That say, thanks for your feedback and I'll try to improve the upstream documentation, in my spare time, as always.

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [camel-kamelets] djencks commented on issue #673: Kamelet for ElasticSearch Sink provides no mechanism to set indexId or ce-indexId

Posted by GitBox <gi...@apache.org>.

djencks commented on issue #673:
URL: https://github.com/apache/camel-kamelets/issues/673#issuecomment-1006402456

OK, so I read https://camel.apache.org/camel-k/next/kamelets/kamelets-user.html and have perhaps some clues and some questions.
- I think it would be helpful to directly link to the user guide from the kamelets catalog index page, something like "consult the user guide for information about how to use these".
- AFAICT the user guide doesn't mention actions. IIUC these have both a source and sink, so you can string them together. An example in the user guide would be great.
- The kamelets in the catalog have what appear to be toy sources and sinks. Is their function for testing? Mentioning this in the user guide might be helpful.

I also think the description of extract field action is pretty close to useless.... "Extract a field from the body".... and then what? Trying to find the source code behind this one, it appears to be in ` - "github:apache.camel-kamelets:camel-kamelets-utils:main-SNAPSHOT"` which I can't locate and which by the "main-SNAPSHOT" makes me wonder if it violates the Apache policy against directing people to unreleased code.

Personally, when someone says the docs are useless, I tend to interpret it as "I'm so frustrated I can't figure out how to do what I want!!!" and try to respond to that.

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [camel-kamelets] ReggieCarey commented on issue #673: Kamelet for ElasticSearch Sink provides no mechanism to set indexId or ce-indexId

Posted by GitBox <gi...@apache.org>.

ReggieCarey commented on issue #673:
URL: https://github.com/apache/camel-kamelets/issues/673#issuecomment-1006941745


   @oscerd,
   
   From my testing and reading of the code at camel-kamelets/library/camel-kamelets-utils/src/main/java/org/apache/camel/kamelets/utils/transform/
   
   **extract-field-action** - Replaces the json cloud-event payload with the contents of an entry in that payload.  It's not clear what syntax to use to step into a json object/array to target a particular nested or array based entry. (JQ?)
   > DOCS: Extract a field from the body
   
   **insert-field-action** - Adds a hard coded field:value to a json cloud-event payload - its not clear what syntax is used to specify some location in the json payload where the field should be inserted. (JQ?)
   > DOCS: Adds an header with a constant value to the message in transit
   
   **insert-header-action** - Adds a hard coded header:value to a cloud-event header. There does not appear to be a bean associated with this so its implementation is a little harder to find.
   > DOCS: No headers mapping supported, only constant values.
   
   ### ??BUG?? ###
   The value fields in the above state they are constant values.  One would assume that any legal json string would be valid for the "value".  This is not the case.  There is some (inappropriate?) parsing of the value field.  Strings like "{{ xxx }}" and "${ xyzzy }" should be valid values but cause a failure in the integration. If one understands what's going on with parsing, one could inject malicious code blocks into that expansion - I haven't found all of the implementing code so I do not know the extent to what gets parsed and how.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [camel-kamelets] oscerd commented on issue #673: Kamelet for ElasticSearch Sink provides no mechanism to set indexId or ce-indexId

Posted by GitBox <gi...@apache.org>.

oscerd commented on issue #673:
URL: https://github.com/apache/camel-kamelets/issues/673#issuecomment-1007217734


   I'm not personally offended by anything you said. My aim is to defend the work we're doing here every day, because I do believe Camel exists for his contributors, occasional contributors or regular contributors is the same.
   
   If I would be a newcomer in the open source world and I would read some of the comments in this issue about documentation that sucks, people from the PMC saying "yeah, right, I'm frustrated" etc. I would stay away, because it looks toxic and it's not. 
   
    You're right about 0.5.x, I'll fix it there too, thanks.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [camel-kamelets] oscerd commented on issue #673: Kamelet for ElasticSearch Sink provides no mechanism to set indexId or ce-indexId

Posted by GitBox <gi...@apache.org>.

oscerd commented on issue #673:
URL: https://github.com/apache/camel-kamelets/issues/673#issuecomment-1006946601


   The fix for es index is already done https://github.com/apache/camel-kamelets/commit/9ffbe779bcd6ed5d1a8e2a818bda13f2e3973c4f#diff-da079f7d2f69897015758cfb8d5a1b24dc3ba677b559c2c3cafb29042c9bb558 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [camel-kamelets] lburgazzoli edited a comment on issue #673: Kamelet for ElasticSearch Sink provides no mechanism to set indexId or ce-indexId

Posted by GitBox <gi...@apache.org>.

lburgazzoli edited a comment on issue #673:
URL: https://github.com/apache/camel-kamelets/issues/673#issuecomment-1008088423


   
   > 
   > Does the Camel DSL or some subset extend to KameletBindings or is it limited to Kamelets?  How much room for configuration do people have if all they want to do is blend existing Kamelets into a solution?
   > 
   
   No, the goal of KameletBinding (which should be renamed to Binding) is to abstract the underlying engine so we do not expect to add any Camel DSL capability directly to the Binding, if you need such flexibility then you should think about using an Integration where you can use any Camel DSL.
   
   > 
   > I see a need for a generic CloudEvent payload processor Action at some point in the future that gives the ability to read and manipulate any payload not just ones that are JSON encoded.  Maybe it is limited to payloads defined with a json schema or protocol buffer or some such to make accessing the contents more predictable.  It might involve introducing a block of user defined code (groovy or python script block?) added to the kameletbinding yaml that executes in some sort of sandbox.
   > 
   
   Adding scripting capabilities through a Kamelet is pretty simple. 
   
   But I want to point you that this has almost nothing to do with CloudEvents as a CloudEventn is nothing more than an envelope that matches what in Camel is a Message (as Andrea pointed out, there is an issue about a better CliudEvent support in Camel). Camel offers already a way to manipulate Messages (of course) by using a scripting language. 
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [camel-kamelets] oscerd commented on issue #673: Kamelet for ElasticSearch Sink provides no mechanism to set indexId or ce-indexId

Posted by GitBox <gi...@apache.org>.

oscerd commented on issue #673:
URL: https://github.com/apache/camel-kamelets/issues/673#issuecomment-1007204896






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [camel-kamelets] oscerd commented on issue #673: Kamelet for ElasticSearch Sink provides no mechanism to set indexId or ce-indexId

Posted by GitBox <gi...@apache.org>.

oscerd commented on issue #673:
URL: https://github.com/apache/camel-kamelets/issues/673#issuecomment-1006561869


   David, you're not trying to do that, you're always trying to find faults in how this PMC work. This is the only thing frustrating, nothing else 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [camel-kamelets] oscerd commented on issue #673: Kamelet for ElasticSearch Sink provides no mechanism to set indexId or ce-indexId

Posted by GitBox <gi...@apache.org>.

oscerd commented on issue #673:
URL: https://github.com/apache/camel-kamelets/issues/673#issuecomment-1007204896


   On 0.5.x the kamelets where still in the main folder, so links should be ok.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [camel-kamelets] oscerd commented on issue #673: Kamelet for ElasticSearch Sink provides no mechanism to set indexId or ce-indexId

Posted by GitBox <gi...@apache.org>.

oscerd commented on issue #673:
URL: https://github.com/apache/camel-kamelets/issues/673#issuecomment-1007422146


   With https://github.com/apache/camel-kamelets/pull/681
   
   You should be able to set headerOutput as parameter and you'll get whatever value you want to extract into the CamelKameletsExtractFieldName header. 
   
   So you could use the header in the subsequent steps. I'm going through the actions for updating and improving some information


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [camel-kamelets] oscerd commented on issue #673: Kamelet for ElasticSearch Sink provides no mechanism to set indexId or ce-indexId

Posted by GitBox <gi...@apache.org>.

oscerd commented on issue #673:
URL: https://github.com/apache/camel-kamelets/issues/673#issuecomment-1007437499


   This will be available in 0.7.0


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [camel-kamelets] davsclaus commented on issue #673: Kamelet for ElasticSearch Sink provides no mechanism to set indexId or ce-indexId

Posted by GitBox <gi...@apache.org>.

davsclaus commented on issue #673:
URL: https://github.com/apache/camel-kamelets/issues/673#issuecomment-1007937720


   > I think it would be helpful to directly link to the user guide from the kamelets catalog index page, something like "consult the user guide for information about how to use these".
   
   I added this to the page


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [camel-kamelets] djencks commented on issue #673: Kamelet for ElasticSearch Sink provides no mechanism to set indexId or ce-indexId

Posted by GitBox <gi...@apache.org>.

djencks commented on issue #673:
URL: https://github.com/apache/camel-kamelets/issues/673#issuecomment-1007203040






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [camel-kamelets] davsclaus commented on issue #673: Kamelet for ElasticSearch Sink provides no mechanism to set indexId or ce-indexId

Posted by GitBox <gi...@apache.org>.

davsclaus commented on issue #673:
URL: https://github.com/apache/camel-kamelets/issues/673#issuecomment-1007250024






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [camel-kamelets] oscerd commented on issue #673: Kamelet for ElasticSearch Sink provides no mechanism to set indexId or ce-indexId

Posted by GitBox <gi...@apache.org>.

oscerd commented on issue #673:
URL: https://github.com/apache/camel-kamelets/issues/673#issuecomment-1008015965


   > @oscerd Thanks for all of the effort, I think this mod adds a huge amount of capability.
   > 
   > But lots of questions and some suggestions:
   > 
   > I'm curious about what it implies to use "Simple" and not Constant for the value parameter binding. This would be very handy in the docs. At a high level, I think it allows for strings with embedded expansions "${some_variable}" but that's just an assumption on my part. Could you add into the documentation what variables we have access to in this expansion and their types (Map, Array, Object, etc.) , e.g. `header` as in "${header[ce-source]}", "${header.ce-source}".
   > 
   
   You could have an idea of how simple works and what is supported from here: https://camel.apache.org/components/3.14.x/languages/simple-language.html
   
   I can add a link in the description about simple language and maybe add some examples somewhere. Yeah.
   
   > I see a potential problem down the road with using a single fixed header (CamelKameletsExtractFieldName). What happens when I need to extract more than one thing to be placed in the header?
   > 
   
   For the header name it makes sense, need to think about a possibile good way of sanitizing. I'd probably also make the sanitize logic optional, like having headerExistenceCheck true or false. Not sure.
   
   I think you're meaning to concatenate more extract-field-action steps and then concatenate the headers when you've done in a new header to use down the route. In that case, yes, it's not a good design what I've done, because you'll end up to have the same header overriden. Ok, I'll check this scenario too.
   
   > Could you do something like this? Note the "field" as well as alternate use of "headerOutput". If we know the payload is JSON, then "field" needs to be interpreted to identify any entry in that json. Ideally this would use a common syntax such as that used by [JQ](https://stedolan.github.io/jq/)
   > 
   > B.T.W. what happens if the thing requested in "field" is not a simple type (string, boolean, number)?
   
   Well, if you have something like a complex type, what you'll get will always be a JSON (just a subset of the original one), with that, you could for example use deserialize-action and transform the JSON into a POJO instance. But in the original design, what is going through actions is always a JSON. By the way maybe adding some examples will be better.
   
   > 
   > ```
   > - route:
   >     from:
   >       uri: "kamelet:kafka-source"
   >       ...
   >     steps:
   >       - to: 
   >           uri: "kamelet:extract-field-action"
   >           parameters:
   >             field: "endUser[3].familyName"
   >             headerOutput: "FAMILY_NAME"
   >       - to: 
   >           uri: "kamelet:extract-field-action"
   >           parameters:
   >             field: "endUser[3].givenName"
   >             headerOutput: "GIVEN_NAME"
   >       - to: 
   >           uri: "kamelet:insert-header-action"
   >           parameters:
   >             name: "ce-indexId"
   >             value: "${header.FAMILY_NAME}_${header.GIVEN_NAME}"
   > or
   >             value: "${header[FAMILY_NAME]}_${header[GIVEN_NAME]}"
   >       - to: 
   >           uri: "kamelet:elasticsearch-index-sink"
   >           ...
   > ```
   > 
   > If my understanding of "Simple" is correct, then in a **KameletBinding** one could establish the headerOutputs in steps A and B and then combine the headers in a subsequent step C as per `value: "${header.FAMILY_NAME}_${header.GIVEN_NAME}"`. Am I understanding the intent properly?
   
   Yes, you could do something like that, but first we need to sanitize the headerName in some way.
   > 
   > Does the Camel DSL or some subset extend to KameletBindings or is it limited to Kamelets? How much room for configuration do people have if all they want to do is blend existing Kamelets into a solution?
   > 
   
   In the beginning Kamelet were only part of camel-k, so you could use the kameletBinding as part of a camel-k integration.
   Later we started to support Kamelet as a plain Camel component and you could do something like from(kamelet:aws-s3).to(kamelet:kafka-sink) even outside camel-k and cloud.
   
   Later we added the support of kamelet into a Jbang extension, still experimental.
   
   So you can use both camel-k, through KameletBinding yaml or you can use camel-kamelet-main through a Camel yaml route using the camel-kamelet component or you could even use the camel-jbang extension for a fast feedback about the yaml route based on kamelets from catalog you're using. But this needs to be defined in the documentation of catalog, camel-kamelets, camel-kamelet-main and camel-jbang a bit better.
   
   > SPECULATION: I see a need for a generic CloudEvent payload processor Action at some point in the future that gives the ability to read and manipulate any payload not just ones that are JSON encoded. Maybe it is limited to payloads defined with a json schema or protocol buffer or some such to make accessing the contents more predictable. It might involve introducing a block of user defined code (groovy or python script block?) added to the kameletbinding yaml that executes in some sort of sandbox.
   
   There were an issue in Camel JIRA about a camel-cloudevent compoonent. But I don't remember if we had some developments around this.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [camel-kamelets] oscerd commented on issue #673: Kamelet for ElasticSearch Sink provides no mechanism to set indexId or ce-indexId

Posted by GitBox <gi...@apache.org>.

oscerd commented on issue #673:
URL: https://github.com/apache/camel-kamelets/issues/673#issuecomment-1006774975


   This is not a product. It's an open source project.
   
   Also there is no camel-k release 3.11.5. What you are refer too is plain camel version, and the documentation you need to refer to is different.
   
   Again there is no product here. You're using something for free here. So you need to respect the work done here.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [camel-kamelets] oscerd commented on issue #673: Kamelet for ElasticSearch Sink provides no mechanism to set indexId or ce-indexId

Posted by GitBox <gi...@apache.org>.

oscerd commented on issue #673:
URL: https://github.com/apache/camel-kamelets/issues/673#issuecomment-1007432546


   With a little update to insert-header action you should be able to do something like
   
   ```
   - route:
       from:
         uri: "kamelet:kafka-source"
         ...
       steps:
         - to: 
             uri: "kamelet:extract-field-action"
             parameters:
               field: "username"
               headerOutput: true
         - to: 
             uri: "kamelet:insert-header-action"
             parameters:
               name: "ce-indexId"
               value: "${header.CamelKameletsExtractFieldName}"
         - to: 
             uri: "kamelet:elasticsearch-index-sink"
             ...
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [camel-kamelets] djencks commented on issue #673: Kamelet for ElasticSearch Sink provides no mechanism to set indexId or ce-indexId

Posted by GitBox <gi...@apache.org>.

djencks commented on issue #673:
URL: https://github.com/apache/camel-kamelets/issues/673#issuecomment-1007203040

   Thanks for fixing the source links. I don’t see an update for 0.5.x?  I’m a bit surprised the link checker didn’t find the problems.

   I’m really glad to discover my fears about the kamelet dependencies linking to unreleased code are unfounded, although since the 0.6.x branch doesn’t appear to have been updated after release the kamelet dependencies on the branch currently do point to snapshots… but the poms also say the version is main-snapshot.

   I apologize for repeatedly discovering something about the processes here that worries me and, after not being able to discover what the actual process is, expressing my doubts that it’s in line with apache polices.  I can see that that’s probably rather infuriating.  I’ll see if I can find another way of asking for information; what I’ve been doing is bad for everyone.

   If there’s something else I’ve said that you are offended about, please let me know what it is.  I’m trying to contribute, not cause problems.

   I think camel is a pretty amazing thing, which is why I’m spending so much time working on the docs and trying to get them to make the relationship between the different parts clearer. 

   David Jencks

   > On Jan 6, 2022, at 9:46 PM, Andrea Cosentino ***@***.***> wrote:
   > 
   > 
   > You weren't asking for this fix, but for something else, using the extract action for setting the indexId from the json routing in the integration. This is not a solution, it's just a workaround.
   > 
   > Thanks for reporting about the links, I'll take care of fixing them. Maybe there were some updates when we changed the way the documentation generation with some PRs from David.
   > 
   > I will always speak up when the work from this community and from the oldest contributors will be underrated and I really wish that members of this PMC do the same, because in the last weeks I just seen attempts to criticize the work done and find faults in what it is done (if someone feel I'm talking about him, yes, I'm talking about you, David).
   > 
   > 20 hours ago I couldn't answer about the fix, because your problem was and it's still different.
   > 
   > The 100's hours of time you consumed, it's 500 hours of work done from contributors around Kamelets and camel-k. The fact that you don't know the code or you have troubles finding where the code is or how it is working, it's part of the game. Camel is a really complex ecosystem, documenting everything will always be impossible, given the number of real contributors working regularly and with passion on this.
   > 
   > I'll take comments like yours as offence, because you and many others, consider, ALWAYS, our time for granted, which it's wrong and unpleasant to read.
   > 
   > That say, thanks for your feedback and I'll trying to improve the upstream documentation, in my spare time, as always.
   > 
   > —
   > Reply to this email directly, view it on GitHub <https://github.com/apache/camel-kamelets/issues/673#issuecomment-1007157005>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAELDXXJQ5NBY6C4NA4RGODUUZ42HANCNFSM5LLAMB2A>.
   > You are receiving this because you commented.
   > 

-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [camel-kamelets] ReggieCarey commented on issue #673: Kamelet for ElasticSearch Sink provides no mechanism to set indexId or ce-indexId

Posted by GitBox <gi...@apache.org>.

ReggieCarey commented on issue #673:
URL: https://github.com/apache/camel-kamelets/issues/673#issuecomment-1007527212

@oscerd Thanks for all of the effort, I think this mod adds a huge amount of capability.

But lots of questions and some suggestions:

I see a potential problem down the road with using a single fixed header (CamelKameletsExtractFieldName). What happens when I need to extract more than one thing to be placed in the header?

B.T.W. what happens if the thing requested in "field" is not a simple type (string, boolean, number)?

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org