You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2021/11/13 11:16:32 UTC
[GitHub] [incubator-doris] morningman opened a new issue #7113: [Feature] Show offset lag for routine load job
morningman opened a new issue #7113:
URL: https://github.com/apache/incubator-doris/issues/7113
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues.
### Description
Show the lag of each kafka offset in `show routine load` stmt's result.
So that we can easily monitor the status of routine load job
### Use case
```
mysql> show routine load\G
*************************** 1. row ***************************
Id: 10013
Name: test4
CreateTime: 2021-11-13 18:59:01
PauseTime: NULL
EndTime: NULL
DbName: default_cluster:db1
TableName: tbl1
State: RUNNING
DataSourceType: KAFKA
CurrentTaskNum: 1
JobProperties: {"timezone":"Asia/Shanghai","send_batch_parallelism":"1","columnSeparator":"','","lineDelimiter":"\n","maxBatchSizeBytes":"209715200","exec_mem_limit":"2147483648","strict_mode":"false","jsonpaths":"","currentTaskConcurrentNum":"1","fuzzy_parse":"false","partitions":"*","columnToColumnExpr":"k1,k2","maxBatchIntervalS":"20","whereExpr":"*","precedingFilter":"*","mergeType":"APPEND","format":"csv","json_root":"","deleteCondition":"*","desireTaskConcurrentNum":"3","maxErrorNum":"0","strip_outer_array":"false","execMemLimit":"2147483648","num_as_string":"false","maxBatchRows":"300000"}
DataSourceProperties: {"topic":"test","currentKafkaPartitions":"0","brokerList":"xxxxx:9093"}
CustomProperties: {"kafka_default_offsets":"OFFSET_BEGINNING","group.id":"test4"}
Statistic: {"receivedBytes":9,"runningTxns":[],"errorRows":0,"committedTaskNum":1,"loadedRows":3,"loadRowsRate":0,"abortedTaskNum":0,"errorRowsAfterResumed":0,"totalRows":3,"unselectedRows":0,"receivedBytesRate":0,"taskExecuteTimeMs":20058}
Progress: {"0":"2"}
Lag: {"0":1}
ReasonOfStateChanged:
ErrorLogUrls:
OtherMsg:
1 row in set (0.02 sec)
```
the `Lag: {"0":1}` shows there is 1 offset behind the latest offset of partition 0.
### Related issues
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #7113: [Feature] Show offset lag for routine load job
Posted by GitBox <gi...@apache.org>.
morningman closed issue #7113:
URL: https://github.com/apache/incubator-doris/issues/7113
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org