You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by "AishD3 (via GitHub)" <gi...@apache.org> on 2024/03/07 07:59:00 UTC
[I] File name format and File Format in Azure Data Lake Sink Connector [camel-kafka-connector]
AishD3 opened a new issue, #1604:
URL: https://github.com/apache/camel-kafka-connector/issues/1604
In Azure Datalake Sink connector , when we move events from Kafka to Gen 2 data lake, Is it possible to configure file name format and file type format like Avro, Json , Parquet in connector configurations.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] File name format and File Format in Azure Data Lake Sink Connector [camel-kafka-connector]
Posted by "AishD3 (via GitHub)" <gi...@apache.org>.
AishD3 commented on issue #1604:
URL: https://github.com/apache/camel-kafka-connector/issues/1604#issuecomment-2004024165
@oscerd can you give sample connector configs with SinkPojoToSchemaAndStructTransform.java to convert file to Avro
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] File name format and File Format in Azure Data Lake Sink Connector [camel-kafka-connector]
Posted by "AishD3 (via GitHub)" <gi...@apache.org>.
AishD3 commented on issue #1604:
URL: https://github.com/apache/camel-kafka-connector/issues/1604#issuecomment-1987961941
Or is it like this in Kafka Header :
<img width="708" alt="image" src="https://github.com/apache/camel-kafka-connector/assets/142221898/5c84e36a-2b9f-4c3c-8fba-150e3283978b">
Is it possible to give an example of Kafka record and header and connector configs , as I am not able to understand the configuration for custom file name. We would like to add - Date , Timestamp on the fileName, whenever file is added to the data lake via this connector. @oscerd
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] File name format and File Format in Azure Data Lake Sink Connector [camel-kafka-connector]
Posted by "oscerd (via GitHub)" <gi...@apache.org>.
oscerd commented on issue #1604:
URL: https://github.com/apache/camel-kafka-connector/issues/1604#issuecomment-1987992230
I don't have to create a reproducer and work on reproducing your use case. It's too much time and I don't have it. Also it requires an Azure Storage account and creating the infra. It's too much stuff to do.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] File name format and File Format in Azure Data Lake Sink Connector [camel-kafka-connector]
Posted by "AishD3 (via GitHub)" <gi...@apache.org>.
AishD3 commented on issue #1604:
URL: https://github.com/apache/camel-kafka-connector/issues/1604#issuecomment-1987827037
@oscerd Kindly requesting guidance on this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] File name format and File Format in Azure Data Lake Sink Connector [camel-kafka-connector]
Posted by "oscerd (via GitHub)" <gi...@apache.org>.
oscerd commented on issue #1604:
URL: https://github.com/apache/camel-kafka-connector/issues/1604#issuecomment-2004026888
On this @valdar is the expert
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] File name format and File Format in Azure Data Lake Sink Connector [camel-kafka-connector]
Posted by "AishD3 (via GitHub)" <gi...@apache.org>.
AishD3 commented on issue #1604:
URL: https://github.com/apache/camel-kafka-connector/issues/1604#issuecomment-2006888798
@oscerd Is it not at all working , and we are almost 2 developers trying to figure this out for one week. Kindly support with more details pls. We want to convert Json event to Avro File in Data Lake.
Is below is right ?
transforms: toavro
transforms.toavro.type: org.apache.camel.kafkaconnector.transforms.SinkPojoToSchemaAndStructTransform$value
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] File name format and File Format in Azure Data Lake Sink Connector [camel-kafka-connector]
Posted by "oscerd (via GitHub)" <gi...@apache.org>.
oscerd commented on issue #1604:
URL: https://github.com/apache/camel-kafka-connector/issues/1604#issuecomment-1987969070
Yes, it should be an header in the kafka record.
I don't know If I'll have time to write an example.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] File name format and File Format in Azure Data Lake Sink Connector [camel-kafka-connector]
Posted by "AishD3 (via GitHub)" <gi...@apache.org>.
AishD3 commented on issue #1604:
URL: https://github.com/apache/camel-kafka-connector/issues/1604#issuecomment-2006232225
@valdar @oscerd kindly requesting your response here. What is sink.pojo.class, what s the value we need to pass to connector for this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] File name format and File Format in Azure Data Lake Sink Connector [camel-kafka-connector]
Posted by "AishD3 (via GitHub)" <gi...@apache.org>.
AishD3 commented on issue #1604:
URL: https://github.com/apache/camel-kafka-connector/issues/1604#issuecomment-2004207212
@valdar Can you pls guide on connector configs of using this transforms, I am always getting "Unable to Initialise the pojo class"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] File name format and File Format in Azure Data Lake Sink Connector [camel-kafka-connector]
Posted by "oscerd (via GitHub)" <gi...@apache.org>.
oscerd commented on issue #1604:
URL: https://github.com/apache/camel-kafka-connector/issues/1604#issuecomment-2006246547
We work on this project in our spare time, we don't have an ETA to answer question and you're pushing a bit too much. This is not a commercial support service. By the way, this is the definition of sink pojo class
**Full qualified class name of the pojo you want your record value converted to**
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] File name format and File Format in Azure Data Lake Sink Connector [camel-kafka-connector]
Posted by "AishD3 (via GitHub)" <gi...@apache.org>.
AishD3 commented on issue #1604:
URL: https://github.com/apache/camel-kafka-connector/issues/1604#issuecomment-1988001516
"CamelHeader.file" works :) , Thank you
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] File name format and File Format in Azure Data Lake Sink Connector [camel-kafka-connector]
Posted by "oscerd (via GitHub)" <gi...@apache.org>.
oscerd commented on issue #1604:
URL: https://github.com/apache/camel-kafka-connector/issues/1604#issuecomment-1987931494
An answered on the other issue:
You need to set an header on record: CamelAzureStorageDataLakeFileName, the header name should be "CamelHeader.CamelAzureStorageDataLakeFileName", it will be transformed while running the sync.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] File name format and File Format in Azure Data Lake Sink Connector [camel-kafka-connector]
Posted by "oscerd (via GitHub)" <gi...@apache.org>.
oscerd commented on issue #1604:
URL: https://github.com/apache/camel-kafka-connector/issues/1604#issuecomment-1987990622
Try by adding an header of this form:
"CamelHeader.file"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] File name format and File Format in Azure Data Lake Sink Connector [camel-kafka-connector]
Posted by "oscerd (via GitHub)" <gi...@apache.org>.
oscerd commented on issue #1604:
URL: https://github.com/apache/camel-kafka-connector/issues/1604#issuecomment-1988002966
This is because of the fact we are now basing the connectors on the Kamelets catalog.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] File name format and File Format in Azure Data Lake Sink Connector [camel-kafka-connector]
Posted by "AishD3 (via GitHub)" <gi...@apache.org>.
AishD3 commented on issue #1604:
URL: https://github.com/apache/camel-kafka-connector/issues/1604#issuecomment-1987994763
<img width="945" alt="image" src="https://github.com/apache/camel-kafka-connector/assets/142221898/1933434e-35aa-44e2-8615-4fbdb120cb36">
posting Kafka record and connector configs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] File name format and File Format in Azure Data Lake Sink Connector [camel-kafka-connector]
Posted by "AishD3 (via GitHub)" <gi...@apache.org>.
AishD3 commented on issue #1604:
URL: https://github.com/apache/camel-kafka-connector/issues/1604#issuecomment-1987976268
@oscerd After adding this config in Kafka header also. The file name is coming with default name like this FD0288760138873-0000000000000001. Can you please guide on this , as we are in need of this on high priority. Kindly requesting your time on this pls. Can you also review the Kafka connector configurations if they are right,
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] File name format and File Format in Azure Data Lake Sink Connector [camel-kafka-connector]
Posted by "AishD3 (via GitHub)" <gi...@apache.org>.
AishD3 commented on issue #1604:
URL: https://github.com/apache/camel-kafka-connector/issues/1604#issuecomment-1987924217
@Oscerd how about fileName - I tried to pass exchangeID is Kafka Header , but its not able to set the file name in Datalake on what I give.
Kafka Header
{
"exchangeId": "adlstest_1000"
}
Connector Configs :
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnector
metadata:
name: adls-sink
labels:
strimzi.io/cluster: cdt-connect-shared-emp
spec:
class: org.apache.camel.kafkaconnector.azurestoragedatalakesink.CamelAzurestoragedatalakesinkSinkConnector
tasksMax: 2
config:
topics: msk.emp.cdtadlsconnectorsink.topic.internal.any.v1,msk.emp.cdtadlsconnectorsink.topic.internal.any.v2
camel.kamelet.azure-storage-datalake-sink.accountName: retinaconnectorpoc
camel.kamelet.azure-storage-datalake-sink.clientId: 328ea5df-d095-4c9f-ba69-79c468792db9
camel.kamelet.azure-storage-datalake-sink.clientSecret: wvvvvvvvvrlanA
camel.kamelet.azure-storage-datalake-sink.tenantId: 05d75c05-fa1a-42e7-9cf1-eb416c396f2d
camel.kamelet.azure-storage-datalake-sink.fileSystemName: test
value.converter.schema.registry.ssl.key.password: ${env:SSL_TRUSTSTORE_PASSWORD}
value.converter.schema.registry.ssl.keystore.location: /opt/kafka/external-configuration/sr-certs/keystore.jks
value.converter.schema.registry.ssl.keystore.password: ${env:SSL_TRUSTSTORE_PASSWORD}
value.converter.schema.registry.ssl.truststore.location: /opt/kafka/external-configuration/sr-certs/truststore.jks
value.converter.schema.registry.ssl.truststore.password: ${env:SSL_TRUSTSTORE_PASSWORD}
value.converter.schema.registry.url: https://cdt-west-vvvvvvvvk-digital.net
value.converter.schemas.enable: false
key.converter.schema.registry.ssl.key.password: ${env:SSL_TRUSTSTORE_PASSWORD}
key.converter.schema.registry.ssl.keystore.location: /opt/kafka/external-configuration/sr-certs/keystore.jks
key.converter.schema.registry.ssl.keystore.password: ${env:SSL_TRUSTSTORE_PASSWORD}
key.converter.schema.registry.ssl.truststore.location: /opt/kafka/external-configuration/sr-certs/truststore.jks
key.converter.schema.registry.ssl.truststore.password: ${env:SSL_TRUSTSTORE_PASSWORD}
key.converter.schema.registry.url: https://cdt-west-vvvvvvvvvdigital.net
key.converter.schemas.enable: false
key.converter: org.apache.kafka.connect.storage.StringConverter
value.converter: org.apache.kafka.connect.storage.StringConverter
camel.aggregation.size: 5
camel.aggregation.timeout: 50000
camelHeader.camelFileName: "adlstestfileanem_1000"
camel.sink.path.directoryName : "/user/app/folder"
camel.sink.endpoint.charset : "UTF-8"
camel.sink.endpoint.fileName : "messages-${date:now:yyyyMMdd}.txt"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] File name format and File Format in Azure Data Lake Sink Connector [camel-kafka-connector]
Posted by "AishD3 (via GitHub)" <gi...@apache.org>.
AishD3 commented on issue #1604:
URL: https://github.com/apache/camel-kafka-connector/issues/1604#issuecomment-2006519062
@valdar @oscerd Still we don't have proper understanding of "Full qualified class name of the pojo you want your record value converted to". Try to give example when free.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] File name format and File Format in Azure Data Lake Sink Connector [camel-kafka-connector]
Posted by "oscerd (via GitHub)" <gi...@apache.org>.
oscerd commented on issue #1604:
URL: https://github.com/apache/camel-kafka-connector/issues/1604#issuecomment-2006305118
As far as I remember this is the only way of transforming directly, other than that you'll need to write your own SMT: https://camel.apache.org/camel-kafka-connector/4.0.x/reference/transformers/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] File name format and File Format in Azure Data Lake Sink Connector [camel-kafka-connector]
Posted by "AishD3 (via GitHub)" <gi...@apache.org>.
AishD3 commented on issue #1604:
URL: https://github.com/apache/camel-kafka-connector/issues/1604#issuecomment-2004933354
@oscerd I just have a small doubt, can you pls guide on this.
key.converter: io.confluent.connect.avro.AvroConverter
value.converter: io.confluent.connect.avro.AvroConverter
transforms: toavro
transforms.toavro.converter.type: value
transforms.toavro.type: org.apache.camel.kafkaconnector.transforms.SinkPojoToSchemaAndStructTransform
transforms.toavro.camel.transformer.sink.pojo.class :
value.converter.enhanced.avro.schema.support: true
connect.meta.data: false
Above is configs with respect to converting to Avro, Here not sure what we need to pass as value for Pojo class.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] File name format and File Format in Azure Data Lake Sink Connector [camel-kafka-connector]
Posted by "oscerd (via GitHub)" <gi...@apache.org>.
oscerd commented on issue #1604:
URL: https://github.com/apache/camel-kafka-connector/issues/1604#issuecomment-2006522938
An example is a full java class name. For example "org.apache.camel.Message" or "java.lang.String"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] File name format and File Format in Azure Data Lake Sink Connector [camel-kafka-connector]
Posted by "oscerd (via GitHub)" <gi...@apache.org>.
oscerd commented on issue #1604:
URL: https://github.com/apache/camel-kafka-connector/issues/1604#issuecomment-1987999984
you don't need to have camelHeader in the config. You just need the header named "CamelHeader.file", also you don't need charse, fileName and keyName, they're doing nothing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] File name format and File Format in Azure Data Lake Sink Connector [camel-kafka-connector]
Posted by "oscerd (via GitHub)" <gi...@apache.org>.
oscerd commented on issue #1604:
URL: https://github.com/apache/camel-kafka-connector/issues/1604#issuecomment-1987859021
For Avro you can transform the pojo into a Generic Avro Record through this SMT https://github.com/apache/camel-kafka-connector/blob/main/core/src/main/java/org/apache/camel/kafkaconnector/transforms/SinkPojoToSchemaAndStructTransform.java
For Json you need to convert the record json in a SMT, you need to write your own SMT. Same for Parquet.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] File name format and File Format in Azure Data Lake Sink Connector [camel-kafka-connector]
Posted by "AishD3 (via GitHub)" <gi...@apache.org>.
AishD3 commented on issue #1604:
URL: https://github.com/apache/camel-kafka-connector/issues/1604#issuecomment-1987948831
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnector
metadata:
name: adls-sink
labels:
strimzi.io/cluster: cdt-connect-shared-emp
spec:
class: org.apache.camel.kafkaconnector.azurestoragedatalakesink.CamelAzurestoragedatalakesinkSinkConnector
tasksMax: 2
config:
topics: msk.emp.cdtadlsconnectorsink.topic.internal.any.v1,msk.emp.cdtadlsconnectorsink.topic.internal.any.v2
camel.kamelet.azure-storage-datalake-sink.accountName: retinaconnectorpoc
camel.kamelet.azure-storage-datalake-sink.clientId: 328dddddddddb9
camel.kamelet.azure-storage-datalake-sink.clientSecret: wro8ddddddddddddodwP.~rlanA
camel.kamelet.azure-storage-datalake-sink.tenantId: 05d75c05-fa1a-42e7-9cf1-eb416c396f2d
camel.kamelet.azure-storage-datalake-sink.fileSystemName: test
value.converter.schema.registry.ssl.key.password: ${env:SSL_TRUSTSTORE_PASSWORD}
value.converter.schema.registry.ssl.keystore.location: /opt/kafka/external-configuration/sr-certs/keystore.jks
value.converter.schema.registry.ssl.keystore.password: ${env:SSL_TRUSTSTORE_PASSWORD}
value.converter.schema.registry.ssl.truststore.location: /opt/kafka/external-configuration/sr-certs/truststore.jks
value.converter.schema.registry.ssl.truststore.password: ${env:SSL_TRUSTSTORE_PASSWORD}
value.converter.schema.registry.url: https://cdt-west-2-shared-emp-kafka-sr.maersk-digital.net
value.converter.schemas.enable: false
key.converter.schema.registry.ssl.key.password: ${env:SSL_TRUSTSTORE_PASSWORD}
key.converter.schema.registry.ssl.keystore.location: /opt/kafka/external-configuration/sr-certs/keystore.jks
key.converter.schema.registry.ssl.keystore.password: ${env:SSL_TRUSTSTORE_PASSWORD}
key.converter.schema.registry.ssl.truststore.location: /opt/kafka/external-configuration/sr-certs/truststore.jks
key.converter.schema.registry.ssl.truststore.password: ${env:SSL_TRUSTSTORE_PASSWORD}
key.converter.schema.registry.url: https://cdt-west-2-shared-emp-kafka-sr.maersk-digital.net
key.converter.schemas.enable: false
key.converter: org.apache.kafka.connect.storage.StringConverter
value.converter: org.apache.kafka.connect.storage.StringConverter
camel.aggregation.size: 5
camel.aggregation.timeout: 50000
camelHeader.camelFileName: "adlstestfileanem_1000"
camel.sink.path.directoryName : "/user/app/folder"
camel.sink.endpoint.charset : "UTF-8"
camel.sink.endpoint.fileName : "${exchangeId}"
camel.sink.endpoint.keyName: "${exchangeId}"
camelHeader.camelAzureStorageDataLakeFileName: "${exchangeId}_adlstest"
Is it like this, I added in last line in connector configs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] File name format and File Format in Azure Data Lake Sink Connector [camel-kafka-connector]
Posted by "AishD3 (via GitHub)" <gi...@apache.org>.
AishD3 commented on issue #1604:
URL: https://github.com/apache/camel-kafka-connector/issues/1604#issuecomment-1988202903
How can we move the files to a particular directory inside the containers. Right now files are directly moved inside container and not going inside directory
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] File name format and File Format in Azure Data Lake Sink Connector [camel-kafka-connector]
Posted by "AishD3 (via GitHub)" <gi...@apache.org>.
AishD3 commented on issue #1604:
URL: https://github.com/apache/camel-kafka-connector/issues/1604#issuecomment-2006292055
@oscerd I understand, We have been trying to explore on this a lot . But it's not working since last one week, that's why. Sorry about the bugging.
If you have sample configuration , it will be really great. This configs and connector runs inside connect cluster and pojo class will be not present inside the connect cluster. Is there any other way to convert these files to Avro in storage account.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] File name format and File Format in Azure Data Lake Sink Connector [camel-kafka-connector]
Posted by "oscerd (via GitHub)" <gi...@apache.org>.
oscerd commented on issue #1604:
URL: https://github.com/apache/camel-kafka-connector/issues/1604#issuecomment-2006898459
Write an SMT for converting the json to avro and use that in your configuration. I don't have time this week to support you more than this and I think I've done a lot.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org