[GitHub] [pulsar] Anonymitaet commented on a change in pull request #5240: [Doc] Add *File source connector guide*

[GitBox <gi...@apache.org>]

Anonymitaet commented on a change in pull request #5240: [Doc] Add *File source connector guide*
URL: https://github.com/apache/pulsar/pull/5240#discussion_r326599774
 
 

 ##########
 File path: site2/docs/io-file-source.md
 ##########
 @@ -0,0 +1,69 @@
+---
+id: io-file
+title: File source connector
+sidebar_label: File source connector
+---
+
+The File source connector pulls messages from files in directories and persists the messages to Pulsar topics.
+
+## Configuration
+
+The configuration of the File source connector has the following properties.
+
+### Property
+
+| Name | Type|Required | Default | Description 
+|------|----------|----------|---------|-------------|
+| `inputDirectory` | String|false|  | The input directory to pull files. |
+| `recurse` | Boolean|false |  | Whether to pull files from subdirectory or not.|
+| `keepFile` |Boolean|false | false | If set to true, the file is not deleted after it is processed, which means the file can be picked up continually. |
+| `fileFilter` | String|false| [^\\.].* | The file whose name matches the given regular expression is picked up. |
+| `pathFilter` | String |false |  | If `recurse` is set to true, the subdirectory whose path matches the given regular expression is scanned. |
+| `minimumFileAge` | Integer|false |  | The minimum age that a file can be processed. <br><br>Any file younger than `minimumFileAge` (according to the last modification date) is ignored. |
+| `maximumFileAge` | Long|false | | The maximum age that a file can be processed. <br><br>Any file older than `maximumFileAge` (according to last modification date) is ignored. |
+| `minimumSize` |Integer| false | | The minimum size (in bytes) that a file can be processed. |
+| `maximumSize` | Double|false || The maximum size (in bytes) that a file can be processed. |
+| `ignoreHiddenFiles` |Boolean| false | | Whether the hidden files should be ignored or not. |
+| `pollingInterval`|Long | false |  | Indicates how long to wait before performing a directory listing. |
+| `numWorkers` | Integer | false | 1 | The number of worker threads that process files.<br><br> This allows you to process a larger number of files concurrently. <br><br>However, setting this to a value greater than 1 makes the data from multiple files mixed in the target topic. |
+
 
 Review comment:
   @tuteng What are their (From line 17-27 except line 19, 20) default values?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services