You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2020/10/14 00:32:42 UTC
[GitHub] [druid] jon-wei commented on a change in pull request #10510: Add docs and integration tests for Auto-compaction snapshot status API
jon-wei commented on a change in pull request #10510:
URL: https://github.com/apache/druid/pull/10510#discussion_r504329613
##########
File path: docs/operations/api-reference.md
##########
@@ -410,6 +410,32 @@ Returns total size and count for each datasource for each interval within given
Returns the total size of segments awaiting compaction for the given dataSource.
This is only valid for dataSource which has compaction enabled.
+##### GET
+
+* `/druid/coordinator/v1/compaction/status`
+
+Returns the status and statistics from the latest auto compaction run of all dataSource which has/had auto compaction enabled.
Review comment:
```suggestion
Returns the status and statistics from the latest auto compaction run of all dataSources which have/had auto compaction enabled.
```
##########
File path: docs/operations/api-reference.md
##########
@@ -410,6 +410,32 @@ Returns total size and count for each datasource for each interval within given
Returns the total size of segments awaiting compaction for the given dataSource.
This is only valid for dataSource which has compaction enabled.
+##### GET
+
+* `/druid/coordinator/v1/compaction/status`
+
+Returns the status and statistics from the latest auto compaction run of all dataSource which has/had auto compaction enabled.
+The response payload includes a list of `latestStatus` object. Each `latestStatus` represent the status for a dataSource (which has/had auto compaction enabled).
Review comment:
```suggestion
The response payload includes a list of `latestStatus` objects. Each `latestStatus` represents the status for a dataSource (which has/had auto compaction enabled).
```
##########
File path: docs/operations/api-reference.md
##########
@@ -410,6 +410,32 @@ Returns total size and count for each datasource for each interval within given
Returns the total size of segments awaiting compaction for the given dataSource.
This is only valid for dataSource which has compaction enabled.
+##### GET
+
+* `/druid/coordinator/v1/compaction/status`
+
+Returns the status and statistics from the latest auto compaction run of all dataSource which has/had auto compaction enabled.
+The response payload includes a list of `latestStatus` object. Each `latestStatus` represent the status for a dataSource (which has/had auto compaction enabled).
+The `latestStatus` object has the following keys:
+* `dataSource`: name of the datasource for this status information
+* `scheduleStatus`: auto compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active auto compaction config submitted otherwise, `NOT_ENABLED`
+* `bytesAwaitingCompaction`: total byte of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction)
+* `bytesCompacted`: total byte of this datasource that is already compacted with the spec set in the auto compaction config.
+* `bytesSkipped`: total byte of this datasource that is skipped (not eligible for auto compaction) by the auto compaction.
+* `segmentCountAwaitingCompaction`: total number of segments of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction)
+* `segmentCountCompacted`: total number of segments of this datasource that is already compacted with the spec set in the auto compaction config.
+* `segmentCountSkipped`: total number of segments of this datasource that is skipped (not eligible for auto compaction) by the auto compaction.
+* `intervalCountAwaitingCompaction`: total number of intervals of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction)
+* `intervalCountCompacted`: total number of intervals of this datasource that is already compacted with the spec set in the auto compaction config.
+* `intervalCountSkipped`: total number of intervals of this datasource that is skipped (not eligible for auto compaction) by the auto compaction.
+
+##### GET
+
+* `/druid/coordinator/v1/compaction/status?dataSource={dataSource}`
+
+Similar to the API `/druid/coordinator/v1/compaction/status` above but filters response to only return information for the {dataSource} given.
+Note that {dataSource} given must has/had auto compaction enabled.
Review comment:
```suggestion
Note that {dataSource} given must have/had auto compaction enabled.
```
##########
File path: docs/operations/metrics.md
##########
@@ -222,13 +222,25 @@ These metrics are for the Druid Coordinator and are reset each time the Coordina
|`segment/dropQueue/count`|Number of segments to drop.|server.|Varies.|
|`segment/size`|Total size of used segments in a data source. Emitted only for data sources to which at least one used segment belongs.|dataSource.|Varies.|
|`segment/count`|Number of used segments belonging to a data source. Emitted only for data sources to which at least one used segment belongs.|dataSource.|< max|
-|`segment/overShadowed/count`|Number of overshadowed segments.||Varies.|
+|`segment/overShadowed/count`|Number of overshadowed segments.| |Varies.|
|`segment/unavailable/count`|Number of segments (not including replicas) left to load until segments that should be loaded in the cluster are available for queries.|dataSource.|0|
|`segment/underReplicated/count`|Number of segments (including replicas) left to load until segments that should be loaded in the cluster are available for queries.|tier, dataSource.|0|
|`tier/historical/count`|Number of available historical nodes in each tier.|tier.|Varies.|
|`tier/replication/factor`|Configured maximum replication factor in each tier.|tier.|Varies.|
|`tier/required/capacity`|Total capacity in bytes required in each tier.|tier.|Varies.|
|`tier/total/capacity`|Total capacity in bytes available in each tier.|tier.|Varies.|
+|`compact/task/count`|Number of task issued in the auto compaction run.| |Varies.|
+|`compactTask/maxSlot/count`|Number of max task slot that can be used for auto compaction tasks in the auto compaction run.| |Varies.|
+|`compactTask/availableSlot/count`|Number of available task slot that can be used for auto compaction tasks in the auto compaction run. (this is max slot minus any currently running compaction task)| |Varies.|
+|`segment/waitCompact/bytes`|Total byte of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction).|datasource.|Varies.|
+|`segment/waitCompact/count`|Total number of segments of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction).|datasource.|Varies.|
Review comment:
```suggestion
|`segment/waitCompact/count`|Total number of segments of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that are eligible for auto compaction).|datasource.|Varies.|
```
##########
File path: docs/operations/metrics.md
##########
@@ -222,13 +222,25 @@ These metrics are for the Druid Coordinator and are reset each time the Coordina
|`segment/dropQueue/count`|Number of segments to drop.|server.|Varies.|
|`segment/size`|Total size of used segments in a data source. Emitted only for data sources to which at least one used segment belongs.|dataSource.|Varies.|
|`segment/count`|Number of used segments belonging to a data source. Emitted only for data sources to which at least one used segment belongs.|dataSource.|< max|
-|`segment/overShadowed/count`|Number of overshadowed segments.||Varies.|
+|`segment/overShadowed/count`|Number of overshadowed segments.| |Varies.|
|`segment/unavailable/count`|Number of segments (not including replicas) left to load until segments that should be loaded in the cluster are available for queries.|dataSource.|0|
|`segment/underReplicated/count`|Number of segments (including replicas) left to load until segments that should be loaded in the cluster are available for queries.|tier, dataSource.|0|
|`tier/historical/count`|Number of available historical nodes in each tier.|tier.|Varies.|
|`tier/replication/factor`|Configured maximum replication factor in each tier.|tier.|Varies.|
|`tier/required/capacity`|Total capacity in bytes required in each tier.|tier.|Varies.|
|`tier/total/capacity`|Total capacity in bytes available in each tier.|tier.|Varies.|
+|`compact/task/count`|Number of task issued in the auto compaction run.| |Varies.|
Review comment:
```suggestion
|`compact/task/count`|Number of tasks issued in the auto compaction run.| |Varies.|
```
##########
File path: docs/operations/api-reference.md
##########
@@ -410,6 +410,32 @@ Returns total size and count for each datasource for each interval within given
Returns the total size of segments awaiting compaction for the given dataSource.
This is only valid for dataSource which has compaction enabled.
+##### GET
+
+* `/druid/coordinator/v1/compaction/status`
+
+Returns the status and statistics from the latest auto compaction run of all dataSource which has/had auto compaction enabled.
+The response payload includes a list of `latestStatus` object. Each `latestStatus` represent the status for a dataSource (which has/had auto compaction enabled).
+The `latestStatus` object has the following keys:
+* `dataSource`: name of the datasource for this status information
+* `scheduleStatus`: auto compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active auto compaction config submitted otherwise, `NOT_ENABLED`
+* `bytesAwaitingCompaction`: total byte of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction)
+* `bytesCompacted`: total byte of this datasource that is already compacted with the spec set in the auto compaction config.
+* `bytesSkipped`: total byte of this datasource that is skipped (not eligible for auto compaction) by the auto compaction.
+* `segmentCountAwaitingCompaction`: total number of segments of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction)
+* `segmentCountCompacted`: total number of segments of this datasource that is already compacted with the spec set in the auto compaction config.
+* `segmentCountSkipped`: total number of segments of this datasource that is skipped (not eligible for auto compaction) by the auto compaction.
+* `intervalCountAwaitingCompaction`: total number of intervals of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction)
Review comment:
```suggestion
* `intervalCountAwaitingCompaction`: total number of intervals of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that are eligible for auto compaction)
```
##########
File path: docs/operations/api-reference.md
##########
@@ -410,6 +410,32 @@ Returns total size and count for each datasource for each interval within given
Returns the total size of segments awaiting compaction for the given dataSource.
This is only valid for dataSource which has compaction enabled.
+##### GET
+
+* `/druid/coordinator/v1/compaction/status`
+
+Returns the status and statistics from the latest auto compaction run of all dataSource which has/had auto compaction enabled.
+The response payload includes a list of `latestStatus` object. Each `latestStatus` represent the status for a dataSource (which has/had auto compaction enabled).
+The `latestStatus` object has the following keys:
+* `dataSource`: name of the datasource for this status information
+* `scheduleStatus`: auto compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active auto compaction config submitted otherwise, `NOT_ENABLED`
+* `bytesAwaitingCompaction`: total byte of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction)
+* `bytesCompacted`: total byte of this datasource that is already compacted with the spec set in the auto compaction config.
+* `bytesSkipped`: total byte of this datasource that is skipped (not eligible for auto compaction) by the auto compaction.
+* `segmentCountAwaitingCompaction`: total number of segments of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction)
+* `segmentCountCompacted`: total number of segments of this datasource that is already compacted with the spec set in the auto compaction config.
Review comment:
```suggestion
* `segmentCountCompacted`: total number of segments of this datasource that are already compacted with the spec set in the auto compaction config.
```
##########
File path: docs/operations/metrics.md
##########
@@ -222,13 +222,25 @@ These metrics are for the Druid Coordinator and are reset each time the Coordina
|`segment/dropQueue/count`|Number of segments to drop.|server.|Varies.|
|`segment/size`|Total size of used segments in a data source. Emitted only for data sources to which at least one used segment belongs.|dataSource.|Varies.|
|`segment/count`|Number of used segments belonging to a data source. Emitted only for data sources to which at least one used segment belongs.|dataSource.|< max|
-|`segment/overShadowed/count`|Number of overshadowed segments.||Varies.|
+|`segment/overShadowed/count`|Number of overshadowed segments.| |Varies.|
|`segment/unavailable/count`|Number of segments (not including replicas) left to load until segments that should be loaded in the cluster are available for queries.|dataSource.|0|
|`segment/underReplicated/count`|Number of segments (including replicas) left to load until segments that should be loaded in the cluster are available for queries.|tier, dataSource.|0|
|`tier/historical/count`|Number of available historical nodes in each tier.|tier.|Varies.|
|`tier/replication/factor`|Configured maximum replication factor in each tier.|tier.|Varies.|
|`tier/required/capacity`|Total capacity in bytes required in each tier.|tier.|Varies.|
|`tier/total/capacity`|Total capacity in bytes available in each tier.|tier.|Varies.|
+|`compact/task/count`|Number of task issued in the auto compaction run.| |Varies.|
+|`compactTask/maxSlot/count`|Number of max task slot that can be used for auto compaction tasks in the auto compaction run.| |Varies.|
+|`compactTask/availableSlot/count`|Number of available task slot that can be used for auto compaction tasks in the auto compaction run. (this is max slot minus any currently running compaction task)| |Varies.|
+|`segment/waitCompact/bytes`|Total byte of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction).|datasource.|Varies.|
+|`segment/waitCompact/count`|Total number of segments of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction).|datasource.|Varies.|
+|`interval/waitCompact/count`|Total number of intervals of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction).|datasource.|Varies.|
+|`segment/compacted/bytes`|Total byte of this datasource that is already compacted with the spec set in the auto compaction config.|datasource.|Varies.|
+|`segment/compacted/count`|Total number of segments of this datasource that is already compacted with the spec set in the auto compaction config.|datasource.|Varies.|
Review comment:
```suggestion
|`segment/compacted/count`|Total number of segments of this datasource that are already compacted with the spec set in the auto compaction config.|datasource.|Varies.|
```
##########
File path: docs/operations/metrics.md
##########
@@ -222,13 +222,25 @@ These metrics are for the Druid Coordinator and are reset each time the Coordina
|`segment/dropQueue/count`|Number of segments to drop.|server.|Varies.|
|`segment/size`|Total size of used segments in a data source. Emitted only for data sources to which at least one used segment belongs.|dataSource.|Varies.|
|`segment/count`|Number of used segments belonging to a data source. Emitted only for data sources to which at least one used segment belongs.|dataSource.|< max|
-|`segment/overShadowed/count`|Number of overshadowed segments.||Varies.|
+|`segment/overShadowed/count`|Number of overshadowed segments.| |Varies.|
|`segment/unavailable/count`|Number of segments (not including replicas) left to load until segments that should be loaded in the cluster are available for queries.|dataSource.|0|
|`segment/underReplicated/count`|Number of segments (including replicas) left to load until segments that should be loaded in the cluster are available for queries.|tier, dataSource.|0|
|`tier/historical/count`|Number of available historical nodes in each tier.|tier.|Varies.|
|`tier/replication/factor`|Configured maximum replication factor in each tier.|tier.|Varies.|
|`tier/required/capacity`|Total capacity in bytes required in each tier.|tier.|Varies.|
|`tier/total/capacity`|Total capacity in bytes available in each tier.|tier.|Varies.|
+|`compact/task/count`|Number of task issued in the auto compaction run.| |Varies.|
+|`compactTask/maxSlot/count`|Number of max task slot that can be used for auto compaction tasks in the auto compaction run.| |Varies.|
+|`compactTask/availableSlot/count`|Number of available task slot that can be used for auto compaction tasks in the auto compaction run. (this is max slot minus any currently running compaction task)| |Varies.|
+|`segment/waitCompact/bytes`|Total byte of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction).|datasource.|Varies.|
Review comment:
```suggestion
|`segment/waitCompact/bytes`|Total bytes of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that are eligible for auto compaction).|datasource.|Varies.|
```
##########
File path: docs/operations/metrics.md
##########
@@ -222,13 +222,25 @@ These metrics are for the Druid Coordinator and are reset each time the Coordina
|`segment/dropQueue/count`|Number of segments to drop.|server.|Varies.|
|`segment/size`|Total size of used segments in a data source. Emitted only for data sources to which at least one used segment belongs.|dataSource.|Varies.|
|`segment/count`|Number of used segments belonging to a data source. Emitted only for data sources to which at least one used segment belongs.|dataSource.|< max|
-|`segment/overShadowed/count`|Number of overshadowed segments.||Varies.|
+|`segment/overShadowed/count`|Number of overshadowed segments.| |Varies.|
|`segment/unavailable/count`|Number of segments (not including replicas) left to load until segments that should be loaded in the cluster are available for queries.|dataSource.|0|
|`segment/underReplicated/count`|Number of segments (including replicas) left to load until segments that should be loaded in the cluster are available for queries.|tier, dataSource.|0|
|`tier/historical/count`|Number of available historical nodes in each tier.|tier.|Varies.|
|`tier/replication/factor`|Configured maximum replication factor in each tier.|tier.|Varies.|
|`tier/required/capacity`|Total capacity in bytes required in each tier.|tier.|Varies.|
|`tier/total/capacity`|Total capacity in bytes available in each tier.|tier.|Varies.|
+|`compact/task/count`|Number of task issued in the auto compaction run.| |Varies.|
+|`compactTask/maxSlot/count`|Number of max task slot that can be used for auto compaction tasks in the auto compaction run.| |Varies.|
Review comment:
```suggestion
|`compactTask/maxSlot/count`|Max number of task slots that can be used for auto compaction tasks in the auto compaction run.| |Varies.|
```
##########
File path: docs/operations/metrics.md
##########
@@ -222,13 +222,25 @@ These metrics are for the Druid Coordinator and are reset each time the Coordina
|`segment/dropQueue/count`|Number of segments to drop.|server.|Varies.|
|`segment/size`|Total size of used segments in a data source. Emitted only for data sources to which at least one used segment belongs.|dataSource.|Varies.|
|`segment/count`|Number of used segments belonging to a data source. Emitted only for data sources to which at least one used segment belongs.|dataSource.|< max|
-|`segment/overShadowed/count`|Number of overshadowed segments.||Varies.|
+|`segment/overShadowed/count`|Number of overshadowed segments.| |Varies.|
|`segment/unavailable/count`|Number of segments (not including replicas) left to load until segments that should be loaded in the cluster are available for queries.|dataSource.|0|
|`segment/underReplicated/count`|Number of segments (including replicas) left to load until segments that should be loaded in the cluster are available for queries.|tier, dataSource.|0|
|`tier/historical/count`|Number of available historical nodes in each tier.|tier.|Varies.|
|`tier/replication/factor`|Configured maximum replication factor in each tier.|tier.|Varies.|
|`tier/required/capacity`|Total capacity in bytes required in each tier.|tier.|Varies.|
|`tier/total/capacity`|Total capacity in bytes available in each tier.|tier.|Varies.|
+|`compact/task/count`|Number of task issued in the auto compaction run.| |Varies.|
+|`compactTask/maxSlot/count`|Number of max task slot that can be used for auto compaction tasks in the auto compaction run.| |Varies.|
+|`compactTask/availableSlot/count`|Number of available task slot that can be used for auto compaction tasks in the auto compaction run. (this is max slot minus any currently running compaction task)| |Varies.|
Review comment:
```suggestion
|`compactTask/availableSlot/count`|Number of available task slots that can be used for auto compaction tasks in the auto compaction run. (this is max slot minus any currently running compaction task)| |Varies.|
```
##########
File path: docs/operations/metrics.md
##########
@@ -222,13 +222,25 @@ These metrics are for the Druid Coordinator and are reset each time the Coordina
|`segment/dropQueue/count`|Number of segments to drop.|server.|Varies.|
|`segment/size`|Total size of used segments in a data source. Emitted only for data sources to which at least one used segment belongs.|dataSource.|Varies.|
|`segment/count`|Number of used segments belonging to a data source. Emitted only for data sources to which at least one used segment belongs.|dataSource.|< max|
-|`segment/overShadowed/count`|Number of overshadowed segments.||Varies.|
+|`segment/overShadowed/count`|Number of overshadowed segments.| |Varies.|
|`segment/unavailable/count`|Number of segments (not including replicas) left to load until segments that should be loaded in the cluster are available for queries.|dataSource.|0|
|`segment/underReplicated/count`|Number of segments (including replicas) left to load until segments that should be loaded in the cluster are available for queries.|tier, dataSource.|0|
|`tier/historical/count`|Number of available historical nodes in each tier.|tier.|Varies.|
|`tier/replication/factor`|Configured maximum replication factor in each tier.|tier.|Varies.|
|`tier/required/capacity`|Total capacity in bytes required in each tier.|tier.|Varies.|
|`tier/total/capacity`|Total capacity in bytes available in each tier.|tier.|Varies.|
+|`compact/task/count`|Number of task issued in the auto compaction run.| |Varies.|
+|`compactTask/maxSlot/count`|Number of max task slot that can be used for auto compaction tasks in the auto compaction run.| |Varies.|
+|`compactTask/availableSlot/count`|Number of available task slot that can be used for auto compaction tasks in the auto compaction run. (this is max slot minus any currently running compaction task)| |Varies.|
+|`segment/waitCompact/bytes`|Total byte of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction).|datasource.|Varies.|
+|`segment/waitCompact/count`|Total number of segments of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction).|datasource.|Varies.|
+|`interval/waitCompact/count`|Total number of intervals of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction).|datasource.|Varies.|
+|`segment/compacted/bytes`|Total byte of this datasource that is already compacted with the spec set in the auto compaction config.|datasource.|Varies.|
+|`segment/compacted/count`|Total number of segments of this datasource that is already compacted with the spec set in the auto compaction config.|datasource.|Varies.|
+|`interval/compacted/count`|Total number of intervals of this datasource that is already compacted with the spec set in the auto compaction config.|datasource.|Varies.|
+|`segment/skipCompact/bytes`|Total byte of this datasource that is skipped (not eligible for auto compaction) by the auto compaction.|datasource.|Varies.|
+|`segment/skipCompact/count`|Total number of segments of this datasource that is skipped (not eligible for auto compaction) by the auto compaction.|datasource.|Varies.|
Review comment:
```suggestion
|`segment/skipCompact/count`|Total number of segments of this datasource that are skipped (not eligible for auto compaction) by the auto compaction.|datasource.|Varies.|
```
##########
File path: docs/operations/metrics.md
##########
@@ -222,13 +222,25 @@ These metrics are for the Druid Coordinator and are reset each time the Coordina
|`segment/dropQueue/count`|Number of segments to drop.|server.|Varies.|
|`segment/size`|Total size of used segments in a data source. Emitted only for data sources to which at least one used segment belongs.|dataSource.|Varies.|
|`segment/count`|Number of used segments belonging to a data source. Emitted only for data sources to which at least one used segment belongs.|dataSource.|< max|
-|`segment/overShadowed/count`|Number of overshadowed segments.||Varies.|
+|`segment/overShadowed/count`|Number of overshadowed segments.| |Varies.|
|`segment/unavailable/count`|Number of segments (not including replicas) left to load until segments that should be loaded in the cluster are available for queries.|dataSource.|0|
|`segment/underReplicated/count`|Number of segments (including replicas) left to load until segments that should be loaded in the cluster are available for queries.|tier, dataSource.|0|
|`tier/historical/count`|Number of available historical nodes in each tier.|tier.|Varies.|
|`tier/replication/factor`|Configured maximum replication factor in each tier.|tier.|Varies.|
|`tier/required/capacity`|Total capacity in bytes required in each tier.|tier.|Varies.|
|`tier/total/capacity`|Total capacity in bytes available in each tier.|tier.|Varies.|
+|`compact/task/count`|Number of task issued in the auto compaction run.| |Varies.|
+|`compactTask/maxSlot/count`|Number of max task slot that can be used for auto compaction tasks in the auto compaction run.| |Varies.|
+|`compactTask/availableSlot/count`|Number of available task slot that can be used for auto compaction tasks in the auto compaction run. (this is max slot minus any currently running compaction task)| |Varies.|
+|`segment/waitCompact/bytes`|Total byte of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction).|datasource.|Varies.|
+|`segment/waitCompact/count`|Total number of segments of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction).|datasource.|Varies.|
+|`interval/waitCompact/count`|Total number of intervals of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction).|datasource.|Varies.|
+|`segment/compacted/bytes`|Total byte of this datasource that is already compacted with the spec set in the auto compaction config.|datasource.|Varies.|
+|`segment/compacted/count`|Total number of segments of this datasource that is already compacted with the spec set in the auto compaction config.|datasource.|Varies.|
+|`interval/compacted/count`|Total number of intervals of this datasource that is already compacted with the spec set in the auto compaction config.|datasource.|Varies.|
+|`segment/skipCompact/bytes`|Total byte of this datasource that is skipped (not eligible for auto compaction) by the auto compaction.|datasource.|Varies.|
Review comment:
```suggestion
|`segment/skipCompact/bytes`|Total bytes of this datasource that are skipped (not eligible for auto compaction) by the auto compaction.|datasource.|Varies.|
```
##########
File path: docs/operations/metrics.md
##########
@@ -222,13 +222,25 @@ These metrics are for the Druid Coordinator and are reset each time the Coordina
|`segment/dropQueue/count`|Number of segments to drop.|server.|Varies.|
|`segment/size`|Total size of used segments in a data source. Emitted only for data sources to which at least one used segment belongs.|dataSource.|Varies.|
|`segment/count`|Number of used segments belonging to a data source. Emitted only for data sources to which at least one used segment belongs.|dataSource.|< max|
-|`segment/overShadowed/count`|Number of overshadowed segments.||Varies.|
+|`segment/overShadowed/count`|Number of overshadowed segments.| |Varies.|
|`segment/unavailable/count`|Number of segments (not including replicas) left to load until segments that should be loaded in the cluster are available for queries.|dataSource.|0|
|`segment/underReplicated/count`|Number of segments (including replicas) left to load until segments that should be loaded in the cluster are available for queries.|tier, dataSource.|0|
|`tier/historical/count`|Number of available historical nodes in each tier.|tier.|Varies.|
|`tier/replication/factor`|Configured maximum replication factor in each tier.|tier.|Varies.|
|`tier/required/capacity`|Total capacity in bytes required in each tier.|tier.|Varies.|
|`tier/total/capacity`|Total capacity in bytes available in each tier.|tier.|Varies.|
+|`compact/task/count`|Number of task issued in the auto compaction run.| |Varies.|
+|`compactTask/maxSlot/count`|Number of max task slot that can be used for auto compaction tasks in the auto compaction run.| |Varies.|
+|`compactTask/availableSlot/count`|Number of available task slot that can be used for auto compaction tasks in the auto compaction run. (this is max slot minus any currently running compaction task)| |Varies.|
+|`segment/waitCompact/bytes`|Total byte of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction).|datasource.|Varies.|
+|`segment/waitCompact/count`|Total number of segments of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction).|datasource.|Varies.|
+|`interval/waitCompact/count`|Total number of intervals of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction).|datasource.|Varies.|
+|`segment/compacted/bytes`|Total byte of this datasource that is already compacted with the spec set in the auto compaction config.|datasource.|Varies.|
+|`segment/compacted/count`|Total number of segments of this datasource that is already compacted with the spec set in the auto compaction config.|datasource.|Varies.|
+|`interval/compacted/count`|Total number of intervals of this datasource that is already compacted with the spec set in the auto compaction config.|datasource.|Varies.|
+|`segment/skipCompact/bytes`|Total byte of this datasource that is skipped (not eligible for auto compaction) by the auto compaction.|datasource.|Varies.|
+|`segment/skipCompact/count`|Total number of segments of this datasource that is skipped (not eligible for auto compaction) by the auto compaction.|datasource.|Varies.|
+|`interval/skipCompact/count`|Total number of intervals of this datasource that is skipped (not eligible for auto compaction) by the auto compaction.|datasource.|Varies.|
Review comment:
```suggestion
|`interval/skipCompact/count`|Total number of intervals of this datasource that are skipped (not eligible for auto compaction) by the auto compaction.|datasource.|Varies.|
```
##########
File path: docs/operations/api-reference.md
##########
@@ -410,6 +410,32 @@ Returns total size and count for each datasource for each interval within given
Returns the total size of segments awaiting compaction for the given dataSource.
This is only valid for dataSource which has compaction enabled.
+##### GET
+
+* `/druid/coordinator/v1/compaction/status`
+
+Returns the status and statistics from the latest auto compaction run of all dataSource which has/had auto compaction enabled.
+The response payload includes a list of `latestStatus` object. Each `latestStatus` represent the status for a dataSource (which has/had auto compaction enabled).
+The `latestStatus` object has the following keys:
+* `dataSource`: name of the datasource for this status information
+* `scheduleStatus`: auto compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active auto compaction config submitted otherwise, `NOT_ENABLED`
+* `bytesAwaitingCompaction`: total byte of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction)
+* `bytesCompacted`: total byte of this datasource that is already compacted with the spec set in the auto compaction config.
+* `bytesSkipped`: total byte of this datasource that is skipped (not eligible for auto compaction) by the auto compaction.
+* `segmentCountAwaitingCompaction`: total number of segments of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction)
+* `segmentCountCompacted`: total number of segments of this datasource that is already compacted with the spec set in the auto compaction config.
+* `segmentCountSkipped`: total number of segments of this datasource that is skipped (not eligible for auto compaction) by the auto compaction.
+* `intervalCountAwaitingCompaction`: total number of intervals of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction)
+* `intervalCountCompacted`: total number of intervals of this datasource that is already compacted with the spec set in the auto compaction config.
+* `intervalCountSkipped`: total number of intervals of this datasource that is skipped (not eligible for auto compaction) by the auto compaction.
Review comment:
```suggestion
* `intervalCountSkipped`: total number of intervals of this datasource that are skipped (not eligible for auto compaction) by the auto compaction.
```
##########
File path: docs/operations/api-reference.md
##########
@@ -410,6 +410,32 @@ Returns total size and count for each datasource for each interval within given
Returns the total size of segments awaiting compaction for the given dataSource.
This is only valid for dataSource which has compaction enabled.
+##### GET
+
+* `/druid/coordinator/v1/compaction/status`
+
+Returns the status and statistics from the latest auto compaction run of all dataSource which has/had auto compaction enabled.
+The response payload includes a list of `latestStatus` object. Each `latestStatus` represent the status for a dataSource (which has/had auto compaction enabled).
+The `latestStatus` object has the following keys:
+* `dataSource`: name of the datasource for this status information
+* `scheduleStatus`: auto compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active auto compaction config submitted otherwise, `NOT_ENABLED`
+* `bytesAwaitingCompaction`: total byte of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction)
Review comment:
```suggestion
* `bytesAwaitingCompaction`: total bytes of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that are eligible for auto compaction)
```
##########
File path: docs/operations/metrics.md
##########
@@ -222,13 +222,25 @@ These metrics are for the Druid Coordinator and are reset each time the Coordina
|`segment/dropQueue/count`|Number of segments to drop.|server.|Varies.|
|`segment/size`|Total size of used segments in a data source. Emitted only for data sources to which at least one used segment belongs.|dataSource.|Varies.|
|`segment/count`|Number of used segments belonging to a data source. Emitted only for data sources to which at least one used segment belongs.|dataSource.|< max|
-|`segment/overShadowed/count`|Number of overshadowed segments.||Varies.|
+|`segment/overShadowed/count`|Number of overshadowed segments.| |Varies.|
|`segment/unavailable/count`|Number of segments (not including replicas) left to load until segments that should be loaded in the cluster are available for queries.|dataSource.|0|
|`segment/underReplicated/count`|Number of segments (including replicas) left to load until segments that should be loaded in the cluster are available for queries.|tier, dataSource.|0|
|`tier/historical/count`|Number of available historical nodes in each tier.|tier.|Varies.|
|`tier/replication/factor`|Configured maximum replication factor in each tier.|tier.|Varies.|
|`tier/required/capacity`|Total capacity in bytes required in each tier.|tier.|Varies.|
|`tier/total/capacity`|Total capacity in bytes available in each tier.|tier.|Varies.|
+|`compact/task/count`|Number of task issued in the auto compaction run.| |Varies.|
+|`compactTask/maxSlot/count`|Number of max task slot that can be used for auto compaction tasks in the auto compaction run.| |Varies.|
+|`compactTask/availableSlot/count`|Number of available task slot that can be used for auto compaction tasks in the auto compaction run. (this is max slot minus any currently running compaction task)| |Varies.|
+|`segment/waitCompact/bytes`|Total byte of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction).|datasource.|Varies.|
+|`segment/waitCompact/count`|Total number of segments of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction).|datasource.|Varies.|
+|`interval/waitCompact/count`|Total number of intervals of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction).|datasource.|Varies.|
Review comment:
```suggestion
|`interval/waitCompact/count`|Total number of intervals of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that are eligible for auto compaction).|datasource.|Varies.|
```
##########
File path: docs/operations/api-reference.md
##########
@@ -410,6 +410,32 @@ Returns total size and count for each datasource for each interval within given
Returns the total size of segments awaiting compaction for the given dataSource.
This is only valid for dataSource which has compaction enabled.
+##### GET
+
+* `/druid/coordinator/v1/compaction/status`
+
+Returns the status and statistics from the latest auto compaction run of all dataSource which has/had auto compaction enabled.
+The response payload includes a list of `latestStatus` object. Each `latestStatus` represent the status for a dataSource (which has/had auto compaction enabled).
+The `latestStatus` object has the following keys:
+* `dataSource`: name of the datasource for this status information
+* `scheduleStatus`: auto compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active auto compaction config submitted otherwise, `NOT_ENABLED`
+* `bytesAwaitingCompaction`: total byte of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction)
+* `bytesCompacted`: total byte of this datasource that is already compacted with the spec set in the auto compaction config.
Review comment:
```suggestion
* `bytesCompacted`: total bytes of this datasource that are already compacted with the spec set in the auto compaction config.
```
##########
File path: docs/operations/metrics.md
##########
@@ -222,13 +222,25 @@ These metrics are for the Druid Coordinator and are reset each time the Coordina
|`segment/dropQueue/count`|Number of segments to drop.|server.|Varies.|
|`segment/size`|Total size of used segments in a data source. Emitted only for data sources to which at least one used segment belongs.|dataSource.|Varies.|
|`segment/count`|Number of used segments belonging to a data source. Emitted only for data sources to which at least one used segment belongs.|dataSource.|< max|
-|`segment/overShadowed/count`|Number of overshadowed segments.||Varies.|
+|`segment/overShadowed/count`|Number of overshadowed segments.| |Varies.|
|`segment/unavailable/count`|Number of segments (not including replicas) left to load until segments that should be loaded in the cluster are available for queries.|dataSource.|0|
|`segment/underReplicated/count`|Number of segments (including replicas) left to load until segments that should be loaded in the cluster are available for queries.|tier, dataSource.|0|
|`tier/historical/count`|Number of available historical nodes in each tier.|tier.|Varies.|
|`tier/replication/factor`|Configured maximum replication factor in each tier.|tier.|Varies.|
|`tier/required/capacity`|Total capacity in bytes required in each tier.|tier.|Varies.|
|`tier/total/capacity`|Total capacity in bytes available in each tier.|tier.|Varies.|
+|`compact/task/count`|Number of task issued in the auto compaction run.| |Varies.|
+|`compactTask/maxSlot/count`|Number of max task slot that can be used for auto compaction tasks in the auto compaction run.| |Varies.|
+|`compactTask/availableSlot/count`|Number of available task slot that can be used for auto compaction tasks in the auto compaction run. (this is max slot minus any currently running compaction task)| |Varies.|
+|`segment/waitCompact/bytes`|Total byte of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction).|datasource.|Varies.|
+|`segment/waitCompact/count`|Total number of segments of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction).|datasource.|Varies.|
+|`interval/waitCompact/count`|Total number of intervals of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction).|datasource.|Varies.|
+|`segment/compacted/bytes`|Total byte of this datasource that is already compacted with the spec set in the auto compaction config.|datasource.|Varies.|
+|`segment/compacted/count`|Total number of segments of this datasource that is already compacted with the spec set in the auto compaction config.|datasource.|Varies.|
+|`interval/compacted/count`|Total number of intervals of this datasource that is already compacted with the spec set in the auto compaction config.|datasource.|Varies.|
Review comment:
```suggestion
|`interval/compacted/count`|Total number of intervals of this datasource that are already compacted with the spec set in the auto compaction config.|datasource.|Varies.|
```
##########
File path: docs/operations/metrics.md
##########
@@ -222,13 +222,25 @@ These metrics are for the Druid Coordinator and are reset each time the Coordina
|`segment/dropQueue/count`|Number of segments to drop.|server.|Varies.|
|`segment/size`|Total size of used segments in a data source. Emitted only for data sources to which at least one used segment belongs.|dataSource.|Varies.|
|`segment/count`|Number of used segments belonging to a data source. Emitted only for data sources to which at least one used segment belongs.|dataSource.|< max|
-|`segment/overShadowed/count`|Number of overshadowed segments.||Varies.|
+|`segment/overShadowed/count`|Number of overshadowed segments.| |Varies.|
|`segment/unavailable/count`|Number of segments (not including replicas) left to load until segments that should be loaded in the cluster are available for queries.|dataSource.|0|
|`segment/underReplicated/count`|Number of segments (including replicas) left to load until segments that should be loaded in the cluster are available for queries.|tier, dataSource.|0|
|`tier/historical/count`|Number of available historical nodes in each tier.|tier.|Varies.|
|`tier/replication/factor`|Configured maximum replication factor in each tier.|tier.|Varies.|
|`tier/required/capacity`|Total capacity in bytes required in each tier.|tier.|Varies.|
|`tier/total/capacity`|Total capacity in bytes available in each tier.|tier.|Varies.|
+|`compact/task/count`|Number of task issued in the auto compaction run.| |Varies.|
+|`compactTask/maxSlot/count`|Number of max task slot that can be used for auto compaction tasks in the auto compaction run.| |Varies.|
+|`compactTask/availableSlot/count`|Number of available task slot that can be used for auto compaction tasks in the auto compaction run. (this is max slot minus any currently running compaction task)| |Varies.|
+|`segment/waitCompact/bytes`|Total byte of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction).|datasource.|Varies.|
+|`segment/waitCompact/count`|Total number of segments of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction).|datasource.|Varies.|
+|`interval/waitCompact/count`|Total number of intervals of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction).|datasource.|Varies.|
+|`segment/compacted/bytes`|Total byte of this datasource that is already compacted with the spec set in the auto compaction config.|datasource.|Varies.|
Review comment:
```suggestion
|`segment/compacted/bytes`|Total bytes of this datasource that are already compacted with the spec set in the auto compaction config.|datasource.|Varies.|
```
##########
File path: docs/operations/api-reference.md
##########
@@ -410,6 +410,32 @@ Returns total size and count for each datasource for each interval within given
Returns the total size of segments awaiting compaction for the given dataSource.
This is only valid for dataSource which has compaction enabled.
+##### GET
+
+* `/druid/coordinator/v1/compaction/status`
+
+Returns the status and statistics from the latest auto compaction run of all dataSource which has/had auto compaction enabled.
+The response payload includes a list of `latestStatus` object. Each `latestStatus` represent the status for a dataSource (which has/had auto compaction enabled).
+The `latestStatus` object has the following keys:
+* `dataSource`: name of the datasource for this status information
+* `scheduleStatus`: auto compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active auto compaction config submitted otherwise, `NOT_ENABLED`
+* `bytesAwaitingCompaction`: total byte of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction)
+* `bytesCompacted`: total byte of this datasource that is already compacted with the spec set in the auto compaction config.
+* `bytesSkipped`: total byte of this datasource that is skipped (not eligible for auto compaction) by the auto compaction.
+* `segmentCountAwaitingCompaction`: total number of segments of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction)
+* `segmentCountCompacted`: total number of segments of this datasource that is already compacted with the spec set in the auto compaction config.
+* `segmentCountSkipped`: total number of segments of this datasource that is skipped (not eligible for auto compaction) by the auto compaction.
+* `intervalCountAwaitingCompaction`: total number of intervals of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction)
+* `intervalCountCompacted`: total number of intervals of this datasource that is already compacted with the spec set in the auto compaction config.
Review comment:
```suggestion
* `intervalCountCompacted`: total number of intervals of this datasource that are already compacted with the spec set in the auto compaction config.
```
##########
File path: docs/operations/api-reference.md
##########
@@ -410,6 +410,32 @@ Returns total size and count for each datasource for each interval within given
Returns the total size of segments awaiting compaction for the given dataSource.
This is only valid for dataSource which has compaction enabled.
+##### GET
+
+* `/druid/coordinator/v1/compaction/status`
+
+Returns the status and statistics from the latest auto compaction run of all dataSource which has/had auto compaction enabled.
+The response payload includes a list of `latestStatus` object. Each `latestStatus` represent the status for a dataSource (which has/had auto compaction enabled).
+The `latestStatus` object has the following keys:
+* `dataSource`: name of the datasource for this status information
+* `scheduleStatus`: auto compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active auto compaction config submitted otherwise, `NOT_ENABLED`
+* `bytesAwaitingCompaction`: total byte of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction)
+* `bytesCompacted`: total byte of this datasource that is already compacted with the spec set in the auto compaction config.
+* `bytesSkipped`: total byte of this datasource that is skipped (not eligible for auto compaction) by the auto compaction.
+* `segmentCountAwaitingCompaction`: total number of segments of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction)
Review comment:
```suggestion
* `segmentCountAwaitingCompaction`: total number of segments of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that are eligible for auto compaction)
```
##########
File path: docs/operations/api-reference.md
##########
@@ -410,6 +410,32 @@ Returns total size and count for each datasource for each interval within given
Returns the total size of segments awaiting compaction for the given dataSource.
This is only valid for dataSource which has compaction enabled.
+##### GET
+
+* `/druid/coordinator/v1/compaction/status`
+
+Returns the status and statistics from the latest auto compaction run of all dataSource which has/had auto compaction enabled.
+The response payload includes a list of `latestStatus` object. Each `latestStatus` represent the status for a dataSource (which has/had auto compaction enabled).
+The `latestStatus` object has the following keys:
+* `dataSource`: name of the datasource for this status information
+* `scheduleStatus`: auto compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active auto compaction config submitted otherwise, `NOT_ENABLED`
+* `bytesAwaitingCompaction`: total byte of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction)
+* `bytesCompacted`: total byte of this datasource that is already compacted with the spec set in the auto compaction config.
+* `bytesSkipped`: total byte of this datasource that is skipped (not eligible for auto compaction) by the auto compaction.
+* `segmentCountAwaitingCompaction`: total number of segments of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction)
+* `segmentCountCompacted`: total number of segments of this datasource that is already compacted with the spec set in the auto compaction config.
+* `segmentCountSkipped`: total number of segments of this datasource that is skipped (not eligible for auto compaction) by the auto compaction.
Review comment:
```suggestion
* `segmentCountSkipped`: total number of segments of this datasource that are skipped (not eligible for auto compaction) by the auto compaction.
```
##########
File path: docs/operations/api-reference.md
##########
@@ -410,6 +410,32 @@ Returns total size and count for each datasource for each interval within given
Returns the total size of segments awaiting compaction for the given dataSource.
This is only valid for dataSource which has compaction enabled.
+##### GET
+
+* `/druid/coordinator/v1/compaction/status`
+
+Returns the status and statistics from the latest auto compaction run of all dataSource which has/had auto compaction enabled.
+The response payload includes a list of `latestStatus` object. Each `latestStatus` represent the status for a dataSource (which has/had auto compaction enabled).
+The `latestStatus` object has the following keys:
+* `dataSource`: name of the datasource for this status information
+* `scheduleStatus`: auto compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active auto compaction config submitted otherwise, `NOT_ENABLED`
+* `bytesAwaitingCompaction`: total byte of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that is eligible for auto compaction)
+* `bytesCompacted`: total byte of this datasource that is already compacted with the spec set in the auto compaction config.
+* `bytesSkipped`: total byte of this datasource that is skipped (not eligible for auto compaction) by the auto compaction.
Review comment:
```suggestion
* `bytesSkipped`: total bytes of this datasource that are skipped (not eligible for auto compaction) by the auto compaction.
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org