[D] Add a file connector sink. [pulsar]

["leukhe (via GitHub)" <gi...@apache.org>]

GitHub user leukhe created a discussion: Add a file connector sink.

First it should be a reverse of the file source connector:
-one file per message
-option to put the date from metadata of message on date of file. 
-atomic write, write to tempdir on same mount, and move file to destination
-if write fails, don't consume message

And later features:
Write to single file, roll to next file after x size/lines/time
Call an executable to process a file. 





GitHub link: https://github.com/apache/pulsar/discussions/22293

----
This is an automatically sent email for commits@pulsar.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@pulsar.apache.org