You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2022/05/10 20:40:34 UTC
[GitHub] [druid] loquisgon commented on a diff in pull request #12488: Emit state of replace and append for native batch tasks
loquisgon commented on code in PR #12488:
URL: https://github.com/apache/druid/pull/12488#discussion_r869661721
##########
docs/operations/metrics.md:
##########
@@ -221,6 +221,41 @@ Note: If the JVM does not support CPU time measurement for the current thread, i
|`worker/taskSlot/total/count`|Number of total task slots on the reporting worker per emission period. This metric is only available if the WorkerTaskCountStatsMonitor module is included.|category, version.|Varies.|
|`worker/taskSlot/used/count`|Number of busy task slots on the reporting worker per emission period. This metric is only available if the WorkerTaskCountStatsMonitor module is included.|category, version.|Varies.|
+## Batch ingestion metrics (Native parallel task)
+
+|Metric|Description|Dimensions|Normal Value|
+|------|-----------|----------|------------|
+|`ingest/batch/append/count`|Count of `1` every time a batch ingestion append job runs|dataSource, taskId, taskType|Always `1`.|
+|`ingest/batch/append/segments/count`|Count of segments created |dataSource, taskId, taskType|At least `1`.|
+|`ingest/batch/ovewrite/count`|Count of `1` every time a batch ingestion overwrite job runs|dataSource, taskId, taskType|Always `1`.|
+|`ingest/batch/ovewrite/segments/count`|Count of segments created |dataSource, taskId, taskType|At least `1`.|
+|`ingest/batch/replace/count`|Count of `1` every time a batch ingestion replace job runs|dataSource, taskId, taskType|Always `1`.|
+|`ingest/batch/replace/segments/count`|Count of segments created |dataSource, taskId, taskType|At least `1`.|
+|`ingest/batch/replace/tombstones/count`|Count of tombstones created in this replace job|dataSource, taskId, taskType|It can be zero when replace could not find any empty time chunks in the input intervals. It cannot be more than number of segments.|
+
+`APPEND`, `OVERWRITE`, and `REPLACE` are decided using the values
+of the `isAppendToExisting` and `isDropExisting` flags in the
+task's `IOConfig` as follows:
+
+|`isAppendToExisting` | `isdDropExisting` | mode |
+|---------------------|-------------------|------|
+`true` | `false` | `APPEND`|
+`true` | `true ` | Invalid combination, exception thrown. |
+`false` | `false` | `OVERWRITE` (this is the default for native batch ingestion). |
+`false` | `true` | `REPLACE`|
Review Comment:
Well, this is to document the batch ingestion mode not the semantics of the method so I am not sure that the explanation for its semantics belongs here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org