You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by "tgib23 (via GitHub)" <gi...@apache.org> on 2023/05/25 07:47:22 UTC
[GitHub] [camel-kafka-connector] tgib23 opened a new issue, #1530: Configs for CamelAzurestorageblobsinkSinkConnector
tgib23 opened a new issue, #1530:
URL: https://github.com/apache/camel-kafka-connector/issues/1530
Hi team,
I'm trying for our use and just confirmed the file is created under the container specified by `camel.kamelet.azure-storage-blob-sink.containerName`.
But is there any way to specify the path under the container, file name, and file extension?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] tgib23 commented on issue #1530: Configs for CamelAzurestorageblobsinkSinkConnector
Posted by "tgib23 (via GitHub)" <gi...@apache.org>.
tgib23 commented on issue #1530:
URL: https://github.com/apache/camel-kafka-connector/issues/1530#issuecomment-1566629584
Hi @oscerd
Do you think the content of this ticket is to be solved as a consequence of CAMEL-19395?
https://issues.apache.org/jira/browse/CAMEL-19395
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] tgib23 commented on issue #1530: Configs for CamelAzurestorageblobsinkSinkConnector
Posted by "tgib23 (via GitHub)" <gi...@apache.org>.
tgib23 commented on issue #1530:
URL: https://github.com/apache/camel-kafka-connector/issues/1530#issuecomment-1594425135
@oscerd
Thanks! it worked, and I can express the folder hierarchies, like `CamelHeader.file:year=2023/month=06/day=16/file_name`.
Where can I find the possible other message headers configurations? I'd like to try them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] oscerd commented on issue #1530: Configs for CamelAzurestorageblobsinkSinkConnector
Posted by "oscerd (via GitHub)" <gi...@apache.org>.
oscerd commented on issue #1530:
URL: https://github.com/apache/camel-kafka-connector/issues/1530#issuecomment-1588766620
I think it is overriden by the Kamelet: can you try with
CamelHeader.file=camel_test ?
Also please note the fileDir header is related to the downloadBlobToFile operation and not the uploadBlockBlob.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] oscerd closed issue #1530: Configs for CamelAzurestorageblobsinkSinkConnector
Posted by "oscerd (via GitHub)" <gi...@apache.org>.
oscerd closed issue #1530: Configs for CamelAzurestorageblobsinkSinkConnector
URL: https://github.com/apache/camel-kafka-connector/issues/1530
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] oscerd commented on issue #1530: Configs for CamelAzurestorageblobsinkSinkConnector
Posted by "oscerd (via GitHub)" <gi...@apache.org>.
oscerd commented on issue #1530:
URL: https://github.com/apache/camel-kafka-connector/issues/1530#issuecomment-1566708024
Need to double check
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] tgib23 commented on issue #1530: Configs for CamelAzurestorageblobsinkSinkConnector
Posted by "tgib23 (via GitHub)" <gi...@apache.org>.
tgib23 commented on issue #1530:
URL: https://github.com/apache/camel-kafka-connector/issues/1530#issuecomment-1566675453
@oscerd
Thanks for clarifying.
Back to my original question, do you know if there is any option to specify the path/filename/extension for Azure Blob Sink?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] tgib23 commented on issue #1530: Configs for CamelAzurestorageblobsinkSinkConnector
Posted by "tgib23 (via GitHub)" <gi...@apache.org>.
tgib23 commented on issue #1530:
URL: https://github.com/apache/camel-kafka-connector/issues/1530#issuecomment-1582087943
@oscerd
I've set the record `CamelHeader.CamelFilename` and `CamelHeader.CamelFilePath` as shown below, but it didn't change the behavior. Anything I'm missing? or any other ideas?
```
[kafka@kafka-team-cluster-kafka-0 kafka]$ ./bin/kafka-console-consumer.sh --topic test-logs --bootstrap-server localhost:9092 --property print.headers=true
CamelHeader.CamelFileName:camel_test,CamelHeader.CamelFilePath:test/year=2023/month=06 {"value":"2023-06-08_07:38:50 test message"}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] tgib23 commented on issue #1530: Configs for CamelAzurestorageblobsinkSinkConnector
Posted by "tgib23 (via GitHub)" <gi...@apache.org>.
tgib23 commented on issue #1530:
URL: https://github.com/apache/camel-kafka-connector/issues/1530#issuecomment-1588661673
Hi @oscerd
Sorry for asking many times, but setting the message headers hasn't had effect and files are uploaded directly under the container with the file name like `E55515147C81BF6-0000000000000001`.
Here are the messages again. Do you have any other ideas? I went through the docs, but couldn't find a clue yet.
```
[kafka@kafka-team-cluster-kafka-0 kafka]$ ./bin/kafka-console-consumer.sh --topic test-logs --bootstrap-server localhost:9092 --property print.headers=true
CamelHeader.CamelAzureStorageBlobOperation:uploadBlockBlob,CamelHeader.CamelAzureStorageBlobBlobName:camel_test,CamelHeader.CamelAzureStorageBlobFileDir:testdir {"value":"2023-06-13_06:41:03 test value"}
CamelHeader.CamelAzureStorageBlobOperation:uploadBlockBlob,CamelHeader.CamelAzureStorageBlobBlobName:camel_test,CamelHeader.CamelAzureStorageBlobFileDir:testdir {"value":"2023-06-13_06:42:03 test value"}
...
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] oscerd commented on issue #1530: Configs for CamelAzurestorageblobsinkSinkConnector
Posted by "oscerd (via GitHub)" <gi...@apache.org>.
oscerd commented on issue #1530:
URL: https://github.com/apache/camel-kafka-connector/issues/1530#issuecomment-1566631644
It's not related
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] oscerd commented on issue #1530: Configs for CamelAzurestorageblobsinkSinkConnector
Posted by "oscerd (via GitHub)" <gi...@apache.org>.
oscerd commented on issue #1530:
URL: https://github.com/apache/camel-kafka-connector/issues/1530#issuecomment-1582281773
It's CamelHeader.CamelAzureStorageBlobBlobName
Please read the documentation: https://camel.apache.org/components/3.20.x/azure-storage-blob-component.html#_endpoint_header_CamelAzureStorageBlobBlobName
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] oscerd commented on issue #1530: Configs for CamelAzurestorageblobsinkSinkConnector
Posted by "oscerd (via GitHub)" <gi...@apache.org>.
oscerd commented on issue #1530:
URL: https://github.com/apache/camel-kafka-connector/issues/1530#issuecomment-1567300581
This could be done by specifying a Camel.file header on each message so you could directory/filename.ext
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] tgib23 commented on issue #1530: Configs for CamelAzurestorageblobsinkSinkConnector
Posted by "tgib23 (via GitHub)" <gi...@apache.org>.
tgib23 commented on issue #1530:
URL: https://github.com/apache/camel-kafka-connector/issues/1530#issuecomment-1569333984
thanks. i'll check and update here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] oscerd commented on issue #1530: Configs for CamelAzurestorageblobsinkSinkConnector
Posted by "oscerd (via GitHub)" <gi...@apache.org>.
oscerd commented on issue #1530:
URL: https://github.com/apache/camel-kafka-connector/issues/1530#issuecomment-1594432369
Hello @tgib23
For the particular use case, the file header is specified in the Kamelet block of code here: https://github.com/apache/camel-kamelets/blob/main/kamelets/azure-storage-blob-sink.kamelet.yaml#L70-L86
So it will be mapped to the following header: CamelAzureStorageBlobBlobName
So this is a case where the Kamelet overrides the value, with a header passed to it.
Talking in general the headers for this component are here: https://camel.apache.org/components/3.20.x/azure-storage-blob-component.html#_message_headers
Don't forget that you always need to prefix CamelHeader. to the header you're going to use.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] oscerd commented on issue #1530: Configs for CamelAzurestorageblobsinkSinkConnector
Posted by "oscerd (via GitHub)" <gi...@apache.org>.
oscerd commented on issue #1530:
URL: https://github.com/apache/camel-kafka-connector/issues/1530#issuecomment-1594433318
I'm closing this, but please re-open or report more issues if needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org