You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@camel.apache.org by "Sergio Gonzalez Meana (Jira)" <ji...@apache.org> on 2022/02/02 23:06:00 UTC
[jira] [Updated] (CAMEL-17590) Camel sftp kafka source connector
[ https://issues.apache.org/jira/browse/CAMEL-17590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sergio Gonzalez Meana updated CAMEL-17590:
------------------------------------------
Description:
Hello
We are trying to read files using the following connector
[https://github.com/apache/camel-kafka-connector-examples/tree/main/sftp/sftp-source]
The connector sends the whole file in one record to kafka (useless on large files).
we have to move big files... aprox 500MB
Is it possible to override the apply method to convert the whole record into small batches?
{{public R apply(R r) {}}
We unsuccessfully tried a couple of approaches with not luck
basically to return a list of newrecords
Thanks in advance
was:
Hello
We are trying to read files using the following connector
[https://github.com/apache/camel-kafka-connector-examples/tree/main/sftp/sftp-source]
The connector sends the whole file in one record to kafka
we need to move a big file aprox 500MB
Is it possible to override the apply method to convert the whole record into small batches?
{{public R apply(R r) {}}
We unsuccessfully tried a couple of things with not luck
basically return a list of newrecords
Thanks in advance
> Camel sftp kafka source connector
> ----------------------------------
>
> Key: CAMEL-17590
> URL: https://issues.apache.org/jira/browse/CAMEL-17590
> Project: Camel
> Issue Type: Bug
> Components: camel-sftp
> Affects Versions: 3.14.0
> Reporter: Sergio Gonzalez Meana
> Priority: Blocker
> Fix For: 3.14.0
>
>
> Hello
>
> We are trying to read files using the following connector
> [https://github.com/apache/camel-kafka-connector-examples/tree/main/sftp/sftp-source]
> The connector sends the whole file in one record to kafka (useless on large files).
> we have to move big files... aprox 500MB
>
> Is it possible to override the apply method to convert the whole record into small batches?
>
> {{public R apply(R r) {}}
> We unsuccessfully tried a couple of approaches with not luck
> basically to return a list of newrecords
>
> Thanks in advance
--
This message was sent by Atlassian Jira
(v8.20.1#820001)