[doris] branch master updated: add the batch interval time of sink in spark connector doc (#16501)

[ji...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/master by this push:
     new 2d7a9c9c11 add the batch interval time of sink in spark connector doc (#16501)
2d7a9c9c11 is described below

commit 2d7a9c9c1157ee9f3fed242f739c609ad8fcdecc
Author: Hong Liu <84...@qq.com>
AuthorDate: Thu Feb 9 08:39:30 2023 +0800

    add the batch interval time of sink in spark connector doc (#16501)
---
 docs/en/docs/ecosystem/spark-doris-connector.md    | 1 +
 docs/zh-CN/docs/ecosystem/spark-doris-connector.md | 1 +
 2 files changed, 2 insertions(+)

diff --git a/docs/en/docs/ecosystem/spark-doris-connector.md b/docs/en/docs/ecosystem/spark-doris-connector.md
index 8a7545f478..56d403f2e7 100644
--- a/docs/en/docs/ecosystem/spark-doris-connector.md
+++ b/docs/en/docs/ecosystem/spark-doris-connector.md
@@ -275,6 +275,7 @@ kafkaSource.selectExpr("CAST(key AS STRING)", "CAST(value as STRING)")
 | sink.properties.*     | --               | The stream load parameters.<br /> <br /> eg:<br /> sink.properties.column_separator' = ','<br /> <br /> |
 | doris.sink.task.partition.size | --                | The number of partitions corresponding to the Writing task. After filtering and other operations, the number of partitions written in Spark RDD may be large, but the number of records corresponding to each Partition is relatively small, resulting in increased writing frequency and waste of computing resources. The smaller this value is set, the less Doris write frequency and less Doris merge pressure. It is generally used with doris. [...]
 | doris.sink.task.use.repartition | false             | Whether to use repartition mode to control the number of partitions written by Doris. The default value is false, and coalesce is used (note: if there is no Spark action before the write, the whole computation will be less parallel). If it is set to true, then repartition is used (note: you can set the final number of partitions at the cost of shuffle). |
+| doris.sink.batch.interval.ms | 50 | The interval time of each batch sink, unit ms. |
 
 ### SQL & Dataframe Configuration
 
diff --git a/docs/zh-CN/docs/ecosystem/spark-doris-connector.md b/docs/zh-CN/docs/ecosystem/spark-doris-connector.md
index 209e53d34e..6c0b8a5c17 100644
--- a/docs/zh-CN/docs/ecosystem/spark-doris-connector.md
+++ b/docs/zh-CN/docs/ecosystem/spark-doris-connector.md
@@ -283,6 +283,7 @@ kafkaSource.selectExpr("CAST(key AS STRING)", "CAST(value as STRING)")
 | sink.properties.*                | --                | Stream Load 的导入参数。<br/>例如:  'sink.properties.column_separator' = ', ' |
 | doris.sink.task.partition.size   | --                | Doris写入任务对应的 Partition 个数。Spark RDD 经过过滤等操作，最后写入的 Partition 数可能会比较大，但每个 Partition 对应的记录数比较少，导致写入频率增加和计算资源浪费。<br/>此数值设置越小，可以降低 Doris 写入频率，减少 Doris 合并压力。该参数配合 doris.sink.task.use.repartition 使用。 |
 | doris.sink.task.use.repartition  | false             | 是否采用 repartition 方式控制 Doris写入 Partition数。默认值为 false，采用 coalesce 方式控制（注意: 如果在写入之前没有 Spark action 算子，可能会导致整个计算并行度降低）。<br/>如果设置为 true，则采用 repartition 方式（注意: 可设置最后 Partition 数，但会额外增加 shuffle 开销）。 |
+| doris.sink.batch.interval.ms | 50 | 每个批次sink的间隔时间，单位 ms。 |
 
 ### SQL 和 Dataframe 专有配置
 


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org