You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by "demo-kratia (via GitHub)" <gi...@apache.org> on 2023/08/02 20:40:58 UTC
[PR] Automatic compaction API documentation refactor (druid)
demo-kratia opened a new pull request, #14740:
URL: https://github.com/apache/druid/pull/14740
Refactor the Automatic compaction API documentation.
Preview: https://druid-git-automatic-compaction-doc-refactor-demo-kratia.vercel.app/docs/api-reference/automatic-compaction-api.html
This PR has:
- [x] been self-reviewed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] Automatic compaction API documentation refactor (druid)
Posted by "vtlim (via GitHub)" <gi...@apache.org>.
vtlim commented on code in PR #14740:
URL: https://github.com/apache/druid/pull/14740#discussion_r1291807426
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,764 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This topic describes the status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. You can configure automatic compaction in the Druid web console or API.
-## Automatic compaction status
+In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments.
+
+## Manage automatic compaction
+
+### Update capacity for compaction tasks
+
+Updates the capacity for compaction tasks. The minimum number of compaction tasks is 1 and the maximum is 2147483652.
Review Comment:
Is this number correct? The `max` below says 2147483647
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,764 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This topic describes the status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. You can configure automatic compaction in the Druid web console or API.
-## Automatic compaction status
+In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments.
+
+## Manage automatic compaction
+
+### Update capacity for compaction tasks
+
+Updates the capacity for compaction tasks. The minimum number of compaction tasks is 1 and the maximum is 2147483652.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction/taskslots</code>
+
+#### Query parameters
+
+To limit the maximum number of compaction tasks, use the optional query parameters `ratio` and `max`:
+
+* `ratio` (optional)
+ * Type: Float
+ * Default: 0.1
+ * Limit the ratio of the total task slots to compaction task slots.
+* `max` (optional)
+ * Type: Int
+ * Default: 2147483647
+ * Limit the maximum number of task slots for compaction tasks.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully updated compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Invalid `max` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000"
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000 HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` message code and an empty response body.
+
+### Create or update automatic compaction configuration
+
+Creates or updates the automatic compaction configuration for a datasource. You pass the automatic compaction as a JSON object in the request body.
+
+The automatic compaction configuration requires only the `dataSource` property. Druid fills all other properties with default values if not specified. See [Automatic compaction dynamic configuration](../configuration/index.md#automatic-compaction-dynamic-configuration) for configuration details.
+
+Note that this endpoint returns an HTTP `200 OK` message code even if the datasource name does not exist.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully submitted auto compaction configuration*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+#### Sample request
+
+The following example creates an automatic compaction configuration for datasource `wikipedia_hour`. This automatic compaction configuration performs compaction on `wikipedia_hour`, resulting in compacted segments that represent a day interval of data.
Review Comment:
```suggestion
The following example creates an automatic compaction configuration for the datasource `wikipedia_hour`, which was ingested with `HOUR` segment granularity. This automatic compaction configuration performs compaction on `wikipedia_hour`, resulting in compacted segments that represent a day interval of data.
```
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,764 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This topic describes the status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. You can configure automatic compaction in the Druid web console or API.
-## Automatic compaction status
+In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments.
+
+## Manage automatic compaction
+
+### Update capacity for compaction tasks
+
+Updates the capacity for compaction tasks. The minimum number of compaction tasks is 1 and the maximum is 2147483652.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction/taskslots</code>
+
+#### Query parameters
+
+To limit the maximum number of compaction tasks, use the optional query parameters `ratio` and `max`:
+
+* `ratio` (optional)
+ * Type: Float
+ * Default: 0.1
+ * Limit the ratio of the total task slots to compaction task slots.
Review Comment:
```suggestion
* Limits the ratio of the total task slots to compaction task slots.
```
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,764 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This topic describes the status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. You can configure automatic compaction in the Druid web console or API.
-## Automatic compaction status
+In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments.
+
+## Manage automatic compaction
+
+### Update capacity for compaction tasks
+
+Updates the capacity for compaction tasks. The minimum number of compaction tasks is 1 and the maximum is 2147483652.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction/taskslots</code>
+
+#### Query parameters
+
+To limit the maximum number of compaction tasks, use the optional query parameters `ratio` and `max`:
+
+* `ratio` (optional)
+ * Type: Float
+ * Default: 0.1
+ * Limit the ratio of the total task slots to compaction task slots.
+* `max` (optional)
+ * Type: Int
+ * Default: 2147483647
+ * Limit the maximum number of task slots for compaction tasks.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully updated compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Invalid `max` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000"
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000 HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` message code and an empty response body.
+
+### Create or update automatic compaction configuration
+
+Creates or updates the automatic compaction configuration for a datasource. You pass the automatic compaction as a JSON object in the request body.
+
+The automatic compaction configuration requires only the `dataSource` property. Druid fills all other properties with default values if not specified. See [Automatic compaction dynamic configuration](../configuration/index.md#automatic-compaction-dynamic-configuration) for configuration details.
+
+Note that this endpoint returns an HTTP `200 OK` message code even if the datasource name does not exist.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully submitted auto compaction configuration*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+#### Sample request
+
+The following example creates an automatic compaction configuration for datasource `wikipedia_hour`. This automatic compaction configuration performs compaction on `wikipedia_hour`, resulting in compacted segments that represent a day interval of data.
+
+In this example:
+
+* `wikipedia_hour` is a datasource with `HOUR` segment granularity.
+* `skipOffsetFromLatest` is set to `PT0S`, meaning that no data is skipped.
+* `partitionsSpec` is set to the default `dynamic`, allowing Druid to dynamically determine the optimal partitioning strategy.
+* `type` is set to `index_parallel`, meaning that parallel indexing is used.
+* `segmentGranularity` is set to `DAY`, meaning that each compacted segment is a day of data.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
-`GET /druid/coordinator/v1/compaction/progress?dataSource={dataSource}`
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"\
+--header 'Content-Type: application/json' \
+--data '{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}'
+```
-Returns the total size of segments awaiting compaction for the given dataSource. The specified dataSource must have [automatic compaction](../data-management/automatic-compaction.md) enabled.
+<!--HTTP-->
-`GET /druid/coordinator/v1/compaction/status`
+```HTTP
+POST /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+Content-Type: application/json
+Content-Length: 281
-Returns the status and statistics from the auto-compaction run of all dataSources which have auto-compaction enabled in the latest run. The response payload includes a list of `latestStatus` objects. Each `latestStatus` represents the status for a dataSource (which has/had auto-compaction enabled).
+{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}
+```
-The `latestStatus` object has the following keys:
-* `dataSource`: name of the datasource for this status information
-* `scheduleStatus`: auto-compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active auto-compaction config submitted. Otherwise, returns `NOT_ENABLED`.
-* `bytesAwaitingCompaction`: total bytes of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `bytesCompacted`: total bytes of this datasource that are already compacted with the spec set in the auto-compaction config
-* `bytesSkipped`: total bytes of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `segmentCountAwaitingCompaction`: total number of segments of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `segmentCountCompacted`: total number of segments of this datasource that are already compacted with the spec set in the auto-compaction config
-* `segmentCountSkipped`: total number of segments of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `intervalCountAwaitingCompaction`: total number of intervals of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `intervalCountCompacted`: total number of intervals of this datasource that are already compacted with the spec set in the auto-compaction config
-* `intervalCountSkipped`: total number of intervals of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
+<!--END_DOCUSAURUS_CODE_TABS-->
-`GET /druid/coordinator/v1/compaction/status?dataSource={dataSource}`
+#### Sample response
-Similar to the API `/druid/coordinator/v1/compaction/status` above but filters response to only return information for the dataSource given.
-The dataSource must have auto-compaction enabled.
+A successful request returns an HTTP `200 OK` message code and an empty response body.
+
+
+### Remove automatic compaction configuration
+
+Removes the automatic compaction configuration for a datasource. This updates the compaction status of the datasource to "Not enabled."
+
+#### URL
+
+<code class="deleteAPI">DELETE</code> <code>/druid/coordinator/v1/config/compaction/:dataSource</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully deleted automatic compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Datasource does not have automatic compaction or invalid datasource name*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request DELETE "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+DELETE /druid/coordinator/v1/config/compaction/wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` message code and an empty response body.
## Automatic compaction configuration
Review Comment:
Since the first heading starts with a verb, what about standardizing the other headings similarly? For example
* View automatic compaction configuration
* Get automatic compaction status
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,764 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This topic describes the status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. You can configure automatic compaction in the Druid web console or API.
-## Automatic compaction status
+In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments.
+
+## Manage automatic compaction
+
+### Update capacity for compaction tasks
+
+Updates the capacity for compaction tasks. The minimum number of compaction tasks is 1 and the maximum is 2147483652.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction/taskslots</code>
+
+#### Query parameters
+
+To limit the maximum number of compaction tasks, use the optional query parameters `ratio` and `max`:
+
+* `ratio` (optional)
+ * Type: Float
+ * Default: 0.1
+ * Limit the ratio of the total task slots to compaction task slots.
+* `max` (optional)
+ * Type: Int
+ * Default: 2147483647
+ * Limit the maximum number of task slots for compaction tasks.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully updated compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Invalid `max` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000"
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000 HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` message code and an empty response body.
+
+### Create or update automatic compaction configuration
+
+Creates or updates the automatic compaction configuration for a datasource. You pass the automatic compaction as a JSON object in the request body.
Review Comment:
```suggestion
Creates or updates the automatic compaction configuration for a datasource. Pass the automatic compaction as a JSON object in the request body.
```
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,764 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This topic describes the status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. You can configure automatic compaction in the Druid web console or API.
-## Automatic compaction status
+In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments.
+
+## Manage automatic compaction
+
+### Update capacity for compaction tasks
+
+Updates the capacity for compaction tasks. The minimum number of compaction tasks is 1 and the maximum is 2147483652.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction/taskslots</code>
+
+#### Query parameters
+
+To limit the maximum number of compaction tasks, use the optional query parameters `ratio` and `max`:
+
+* `ratio` (optional)
+ * Type: Float
+ * Default: 0.1
+ * Limit the ratio of the total task slots to compaction task slots.
+* `max` (optional)
+ * Type: Int
+ * Default: 2147483647
+ * Limit the maximum number of task slots for compaction tasks.
Review Comment:
```suggestion
* Limits the maximum number of task slots for compaction tasks.
```
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,764 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This topic describes the status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. You can configure automatic compaction in the Druid web console or API.
-## Automatic compaction status
+In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments.
+
+## Manage automatic compaction
+
+### Update capacity for compaction tasks
+
+Updates the capacity for compaction tasks. The minimum number of compaction tasks is 1 and the maximum is 2147483652.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction/taskslots</code>
+
+#### Query parameters
+
+To limit the maximum number of compaction tasks, use the optional query parameters `ratio` and `max`:
+
+* `ratio` (optional)
+ * Type: Float
+ * Default: 0.1
+ * Limit the ratio of the total task slots to compaction task slots.
+* `max` (optional)
+ * Type: Int
+ * Default: 2147483647
+ * Limit the maximum number of task slots for compaction tasks.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully updated compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Invalid `max` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000"
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000 HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` message code and an empty response body.
+
+### Create or update automatic compaction configuration
Review Comment:
I think this request should go at the top since it will probably be used more than the update capacity request. WDYT? Maybe order by
* Create
* Remove
* Update capacity
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,764 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This topic describes the status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. You can configure automatic compaction in the Druid web console or API.
-## Automatic compaction status
+In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments.
+
+## Manage automatic compaction
+
+### Update capacity for compaction tasks
+
+Updates the capacity for compaction tasks. The minimum number of compaction tasks is 1 and the maximum is 2147483652.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction/taskslots</code>
+
+#### Query parameters
+
+To limit the maximum number of compaction tasks, use the optional query parameters `ratio` and `max`:
+
+* `ratio` (optional)
+ * Type: Float
+ * Default: 0.1
+ * Limit the ratio of the total task slots to compaction task slots.
+* `max` (optional)
+ * Type: Int
+ * Default: 2147483647
+ * Limit the maximum number of task slots for compaction tasks.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully updated compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Invalid `max` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000"
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000 HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` message code and an empty response body.
+
+### Create or update automatic compaction configuration
+
+Creates or updates the automatic compaction configuration for a datasource. You pass the automatic compaction as a JSON object in the request body.
+
+The automatic compaction configuration requires only the `dataSource` property. Druid fills all other properties with default values if not specified. See [Automatic compaction dynamic configuration](../configuration/index.md#automatic-compaction-dynamic-configuration) for configuration details.
+
+Note that this endpoint returns an HTTP `200 OK` message code even if the datasource name does not exist.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully submitted auto compaction configuration*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+#### Sample request
+
+The following example creates an automatic compaction configuration for datasource `wikipedia_hour`. This automatic compaction configuration performs compaction on `wikipedia_hour`, resulting in compacted segments that represent a day interval of data.
+
+In this example:
+
+* `wikipedia_hour` is a datasource with `HOUR` segment granularity.
+* `skipOffsetFromLatest` is set to `PT0S`, meaning that no data is skipped.
+* `partitionsSpec` is set to the default `dynamic`, allowing Druid to dynamically determine the optimal partitioning strategy.
+* `type` is set to `index_parallel`, meaning that parallel indexing is used.
+* `segmentGranularity` is set to `DAY`, meaning that each compacted segment is a day of data.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
-`GET /druid/coordinator/v1/compaction/progress?dataSource={dataSource}`
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"\
+--header 'Content-Type: application/json' \
+--data '{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}'
+```
-Returns the total size of segments awaiting compaction for the given dataSource. The specified dataSource must have [automatic compaction](../data-management/automatic-compaction.md) enabled.
+<!--HTTP-->
-`GET /druid/coordinator/v1/compaction/status`
+```HTTP
+POST /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+Content-Type: application/json
+Content-Length: 281
-Returns the status and statistics from the auto-compaction run of all dataSources which have auto-compaction enabled in the latest run. The response payload includes a list of `latestStatus` objects. Each `latestStatus` represents the status for a dataSource (which has/had auto-compaction enabled).
+{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}
+```
-The `latestStatus` object has the following keys:
-* `dataSource`: name of the datasource for this status information
-* `scheduleStatus`: auto-compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active auto-compaction config submitted. Otherwise, returns `NOT_ENABLED`.
-* `bytesAwaitingCompaction`: total bytes of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `bytesCompacted`: total bytes of this datasource that are already compacted with the spec set in the auto-compaction config
-* `bytesSkipped`: total bytes of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `segmentCountAwaitingCompaction`: total number of segments of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `segmentCountCompacted`: total number of segments of this datasource that are already compacted with the spec set in the auto-compaction config
-* `segmentCountSkipped`: total number of segments of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `intervalCountAwaitingCompaction`: total number of intervals of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `intervalCountCompacted`: total number of intervals of this datasource that are already compacted with the spec set in the auto-compaction config
-* `intervalCountSkipped`: total number of intervals of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
+<!--END_DOCUSAURUS_CODE_TABS-->
-`GET /druid/coordinator/v1/compaction/status?dataSource={dataSource}`
+#### Sample response
-Similar to the API `/druid/coordinator/v1/compaction/status` above but filters response to only return information for the dataSource given.
-The dataSource must have auto-compaction enabled.
+A successful request returns an HTTP `200 OK` message code and an empty response body.
+
+
+### Remove automatic compaction configuration
+
+Removes the automatic compaction configuration for a datasource. This updates the compaction status of the datasource to "Not enabled."
+
+#### URL
+
+<code class="deleteAPI">DELETE</code> <code>/druid/coordinator/v1/config/compaction/:dataSource</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully deleted automatic compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Datasource does not have automatic compaction or invalid datasource name*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request DELETE "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+DELETE /druid/coordinator/v1/config/compaction/wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` message code and an empty response body.
## Automatic compaction configuration
-`GET /druid/coordinator/v1/config/compaction`
+### Get all automatic compaction configurations
+
+Retrieves all automatic compaction configurations. Returns a `compactionConfigs` object containing the active automatic compaction configurations of all datasources.
+
+You can use this endpoint to retrieve `compactionTaskSlotRatio` and `maxCompactionTaskSlots` values for managing resource allocation of compaction tasks.
+
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved automatic compaction configurations*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
-Returns all automatic compaction configs.
+```json
+{
+ "compactionConfigs": [
+ {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ {
+ "dataSource": "wikipedia",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ }
+ ],
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+}
+```
+</details>
-`GET /druid/coordinator/v1/config/compaction/{dataSource}`
+### Get automatic compaction configuration
-Returns an automatic compaction config of a dataSource.
+Retrieves the automatic compaction configuration for a datasource.
-`GET /druid/coordinator/v1/config/compaction/{dataSource}/history?interval={interval}&count={count}`
+#### URL
-Returns the history of the automatic compaction config for a dataSource. Optionally accepts `interval` and `count`
-query string parameters to filter by interval and limit the number of results respectively. If the dataSource does not
-exist or there is no compaction history for the dataSource, an empty list is returned.
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction/:dataSource</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved configuration for datasource*
+
+<!--404 NOT FOUND-->
+
+*Invalid datasource or datasource does not have automatic compaction enabled*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+The following example retrieves the automatic compaction configuration for datasource `wikipedia_hour`.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction/wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
+
+```json
+{
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+}
+```
+</details>
+
+### Get automatic compaction configuration history
+
+Retrieves the history of the automatic compaction configuration for a datasource. Returns an empty list if the datasource does not exist or there is no compaction history for the datasource.
The response contains a list of objects with the following keys:
-* `globalConfig`: A json object containing automatic compaction config that applies to the entire cluster.
-* `compactionConfig`: A json object containing the automatic compaction config for the datasource.
-* `auditInfo`: A json object that contains information about the change made - like `author`, `comment` and `ip`.
+* `globalConfig`: A JSON object containing automatic compaction configuration that applies to the entire cluster.
+* `compactionConfig`: A JSON object containing the automatic compaction configuration for the datasource.
+* `auditInfo`: A JSON object containing information about the change made, such as `author`, `comment` or `ip`.
* `auditTime`: The date and time when the change was made.
-`POST /druid/coordinator/v1/config/compaction/taskslots?ratio={someRatio}&max={someMaxSlots}`
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction/:dataSource/history</code>
+
+#### Query parameters
+* `interval` (optional)
+ * Type: ISO-8601
+ * Limit the results within a specified interval. Use `/` as the delimiter for the interval string.
+* `count` (optional)
+ * Type: Int
+ * Limit the number of results.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved configuration history*
+
+<!--400 BAD REQUEST-->
+
+*Invalid `count` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour/history"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction/wikipedia_hour/history HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
+
+```json
+[
+ {
+ "globalConfig": {
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+ },
+ "compactionConfig": {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "P1D",
+ "tuningConfig": null,
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ "auditInfo": {
+ "author": "",
+ "comment": "",
+ "ip": "127.0.0.1"
+ },
+ "auditTime": "2023-07-31T18:15:19.302Z"
+ },
+ {
+ "globalConfig": {
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+ },
+ "compactionConfig": {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ "auditInfo": {
+ "author": "",
+ "comment": "",
+ "ip": "127.0.0.1"
+ },
+ "auditTime": "2023-07-31T18:16:16.362Z"
+ }
+]
+```
+</details>
+
+## Automatic compaction status
+
+### Get segments awaiting compaction
+
+Returns the total size of segments awaiting compaction for a given datasource. The specified datasource must have automatic compaction enabled.
Review Comment:
If it doesn't, does this request fail, or return an empty response?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] Automatic compaction API documentation refactor (druid)
Posted by "ektravel (via GitHub)" <gi...@apache.org>.
ektravel commented on code in PR #14740:
URL: https://github.com/apache/druid/pull/14740#discussion_r1290635136
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,762 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This topic describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. You can configure automatic compaction in the Druid web console or API.
-## Automatic compaction status
+In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments.
+
+## Manage automatic compaction
+
+### Update capacity for compaction tasks
+
+Update the capacity for compaction tasks. The minimum number of compaction tasks is 1 and the maximum is 2147483652.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction/taskslots</code>
+
+#### Query parameters
+
+To limit the maximum number of compaction tasks, use the optional query parameters `ratio` and `max`:
+
+* `ratio` (optional)
+ * Type: Float
+ * Default: 0.1
+ * Limit the ratio of the total task slots to compaction task slots.
+* `max` (optional)
+ * Type: Int
+ * Default: 2147483647
+ * Limit the maximum number of task slots for compaction tasks.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully updated compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Invalid `max` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000"
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000 HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` message code and an empty response body.
+
+### Create or update automatic compaction configuration
+
+Creates or updates the automatic compaction configuration for a datasource. You pass the automatic compaction as a JSON object in the request body.
+
+Automatic compaction configuration require only the `dataSource` property. Druid fills all other properties with default values if not specified. See [Automatic compaction dynamic configuration](../configuration/index.md#automatic-compaction-dynamic-configuration) for configuration details.
+
+Note that this endpoint returns an HTTP `200 OK` message code even if the datasource name does not exist.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully submitted auto compaction configuration*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+#### Sample request
+
+The following example creates an automatic compaction configuration for datasource `wikipedia_hour`. This automatic compaction configuration performs compaction on `wikipedia_hour`, resulting in compacted segments that represent a day interval of data. In this example:
+
+* `wikipedia_hour` is a datasource with `HOUR` segment granularity.
+* `skipOffsetFromLatest` is set to `PT0S`, meaning that no data is skipped.
+* `partitionsSpec` is set to the default `dynamic`, allowing Druid to dynamically determine the optimal partitioning strategy.
+* `type` is set to `index_parallel`, meaning that parallel indexing is used.
+* `segmentGranularity` is set to `DAY`, meaning that each compacted segment is a day of data.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
-`GET /druid/coordinator/v1/compaction/progress?dataSource={dataSource}`
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"\
+--header 'Content-Type: application/json' \
+--data '{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}'
+```
-Returns the total size of segments awaiting compaction for the given dataSource. The specified dataSource must have [automatic compaction](../data-management/automatic-compaction.md) enabled.
+<!--HTTP-->
-`GET /druid/coordinator/v1/compaction/status`
+```HTTP
+POST /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+Content-Type: application/json
+Content-Length: 281
-Returns the status and statistics from the auto-compaction run of all dataSources which have auto-compaction enabled in the latest run. The response payload includes a list of `latestStatus` objects. Each `latestStatus` represents the status for a dataSource (which has/had auto-compaction enabled).
+{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}
+```
-The `latestStatus` object has the following keys:
-* `dataSource`: name of the datasource for this status information
-* `scheduleStatus`: auto-compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active auto-compaction config submitted. Otherwise, returns `NOT_ENABLED`.
-* `bytesAwaitingCompaction`: total bytes of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `bytesCompacted`: total bytes of this datasource that are already compacted with the spec set in the auto-compaction config
-* `bytesSkipped`: total bytes of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `segmentCountAwaitingCompaction`: total number of segments of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `segmentCountCompacted`: total number of segments of this datasource that are already compacted with the spec set in the auto-compaction config
-* `segmentCountSkipped`: total number of segments of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `intervalCountAwaitingCompaction`: total number of intervals of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `intervalCountCompacted`: total number of intervals of this datasource that are already compacted with the spec set in the auto-compaction config
-* `intervalCountSkipped`: total number of intervals of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
+<!--END_DOCUSAURUS_CODE_TABS-->
-`GET /druid/coordinator/v1/compaction/status?dataSource={dataSource}`
+#### Sample response
-Similar to the API `/druid/coordinator/v1/compaction/status` above but filters response to only return information for the dataSource given.
-The dataSource must have auto-compaction enabled.
+A successful request returns an HTTP `200 OK` message code and an empty response body.
+
+
+### Remove automatic compaction configuration
+
+Removes the automatic compaction configuration for a datasource. This updates the compaction status of the datasource to "Not enabled."
+
+#### URL
+
+<code class="deleteAPI">DELETE</code> <code>/druid/coordinator/v1/config/compaction/:dataSource</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully deleted automatic compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Datasource does not have automatic compaction or invalid datasource name*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request DELETE "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+DELETE /druid/coordinator/v1/config/compaction/wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` message code and an empty response body.
## Automatic compaction configuration
-`GET /druid/coordinator/v1/config/compaction`
+### Get all automatic compaction configurations
+
+Retrieves all automatic compaction configurations. Returns a `compactionConfigs` object containing the active automatic compaction configuration of all datasources.
+
+You can use this endpoint to retrieve `compactionTaskSlotRatio` and `maxCompactionTaskSlots` values for managing resource allocation of compaction tasks.
+
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved automatic compaction configurations*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
-Returns all automatic compaction configs.
+```json
+{
+ "compactionConfigs": [
+ {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ {
+ "dataSource": "wikipedia",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ }
+ ],
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+}
+```
+</details>
-`GET /druid/coordinator/v1/config/compaction/{dataSource}`
+### Get automatic compaction configuration
-Returns an automatic compaction config of a dataSource.
+Retrieves the automatic compaction configuration of a datasource.
-`GET /druid/coordinator/v1/config/compaction/{dataSource}/history?interval={interval}&count={count}`
+#### URL
-Returns the history of the automatic compaction config for a dataSource. Optionally accepts `interval` and `count`
-query string parameters to filter by interval and limit the number of results respectively. If the dataSource does not
-exist or there is no compaction history for the dataSource, an empty list is returned.
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction/:dataSource</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved configuration for datasource*
+
+<!--404 NOT FOUND-->
+
+*Invalid datasource or datasource does not have automatic compaction enabled*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+The following example retrieves the automatic compaction configuration for datasource `wikipedia_hour`.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction/wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
+
+```json
+{
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+}
+```
+</details>
+
+### Get automatic compaction configuration history
+
+Retrieves the history of the automatic compaction configuration for a datasource. Returns an empty list if the datasource does not exist or there is no compaction history for the datasource.
The response contains a list of objects with the following keys:
-* `globalConfig`: A json object containing automatic compaction config that applies to the entire cluster.
-* `compactionConfig`: A json object containing the automatic compaction config for the datasource.
-* `auditInfo`: A json object that contains information about the change made - like `author`, `comment` and `ip`.
+* `globalConfig`: A JSON object containing automatic compaction configuration that applies to the entire cluster.
+* `compactionConfig`: A JSON object containing the automatic compaction configuration for the datasource.
+* `auditInfo`: A JSON object that contains information about the change made, such as `author`, `comment` or `ip`.
* `auditTime`: The date and time when the change was made.
-`POST /druid/coordinator/v1/config/compaction/taskslots?ratio={someRatio}&max={someMaxSlots}`
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction/:dataSource/history</code>
+
+#### Query parameters
+* `interval` (optional)
+ * Type: ISO-8601
+ * Limit the results within a specified interval. Use `/` as the delimiter for the interval string.
+* `count` (optional)
+ * Type: Int
+ * Limit the number of results.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved configuration history*
+
+<!--400 BAD REQUEST-->
+
+*Invalid `count` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour/history"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction/wikipedia_hour/history HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
+
+```json
+[
+ {
+ "globalConfig": {
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+ },
+ "compactionConfig": {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "P1D",
+ "tuningConfig": null,
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ "auditInfo": {
+ "author": "",
+ "comment": "",
+ "ip": "127.0.0.1"
+ },
+ "auditTime": "2023-07-31T18:15:19.302Z"
+ },
+ {
+ "globalConfig": {
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+ },
+ "compactionConfig": {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ "auditInfo": {
+ "author": "",
+ "comment": "",
+ "ip": "127.0.0.1"
+ },
+ "auditTime": "2023-07-31T18:16:16.362Z"
+ }
+]
+```
+</details>
+
+## Automatic compaction status
+
+### Get segments awaiting compaction
+
+Returns the total size of segments awaiting compaction for a given datasource. The specified datasource must have automatic compaction enabled.
+
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/compaction/progress?dataSource=:dataSource</code>
+
+#### Query parameter
+* `dataSource` (required)
+ * Type: String
+ * Name of the datasource for this status information
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved segment size awaiting compaction*
+
+<!--404 NOT FOUND-->
+
+*Unknown datasource name or datasource does not have automatic compaction enabled*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+The following example retrieves the remaining segments to be compacted for datasource `wikipedia_hour`.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/compaction/progress?dataSource=wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/compaction/progress?dataSource=wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
+
+```json
+{
+ "remainingSegmentSize": 7615837
+}
+```
+</details>
+
+
+### Get compaction status and statistics
+
+Retrieves an array of `latestStatus` objects representing the status and statistics from the latest automatic compaction run for all datasources with automatic compaction enabled.
+
+The `latestStatus` object has the following properties:
+* `dataSource`: Name of the datasource for this status information.
+* `scheduleStatus`: Automatic compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the datasource has an active automatic compaction configuration submitted. Otherwise, returns `NOT_ENABLED`.
+* `bytesAwaitingCompaction`: Total bytes of this datasource waiting to be compacted by the automatic compaction (only consider intervals/segments that are eligible for automatic compaction).
+* `bytesCompacted`: Total bytes of this datasource that are already compacted with the spec set in the automatic compaction config.
Review Comment:
```suggestion
* `bytesCompacted`: Total bytes of this datasource that are already compacted with the spec set in the automatic compaction configuration.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] Automatic compaction API documentation refactor (druid)
Posted by "vtlim (via GitHub)" <gi...@apache.org>.
vtlim merged PR #14740:
URL: https://github.com/apache/druid/pull/14740
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] Automatic compaction API documentation refactor (druid)
Posted by "ektravel (via GitHub)" <gi...@apache.org>.
ektravel commented on code in PR #14740:
URL: https://github.com/apache/druid/pull/14740#discussion_r1290618418
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,762 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This topic describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. You can configure automatic compaction in the Druid web console or API.
-## Automatic compaction status
+In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments.
+
+## Manage automatic compaction
+
+### Update capacity for compaction tasks
+
+Update the capacity for compaction tasks. The minimum number of compaction tasks is 1 and the maximum is 2147483652.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction/taskslots</code>
+
+#### Query parameters
+
+To limit the maximum number of compaction tasks, use the optional query parameters `ratio` and `max`:
+
+* `ratio` (optional)
+ * Type: Float
+ * Default: 0.1
+ * Limit the ratio of the total task slots to compaction task slots.
+* `max` (optional)
+ * Type: Int
+ * Default: 2147483647
+ * Limit the maximum number of task slots for compaction tasks.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully updated compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Invalid `max` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000"
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000 HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` message code and an empty response body.
+
+### Create or update automatic compaction configuration
+
+Creates or updates the automatic compaction configuration for a datasource. You pass the automatic compaction as a JSON object in the request body.
+
+Automatic compaction configuration require only the `dataSource` property. Druid fills all other properties with default values if not specified. See [Automatic compaction dynamic configuration](../configuration/index.md#automatic-compaction-dynamic-configuration) for configuration details.
+
+Note that this endpoint returns an HTTP `200 OK` message code even if the datasource name does not exist.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully submitted auto compaction configuration*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+#### Sample request
+
+The following example creates an automatic compaction configuration for datasource `wikipedia_hour`. This automatic compaction configuration performs compaction on `wikipedia_hour`, resulting in compacted segments that represent a day interval of data. In this example:
Review Comment:
```suggestion
The following example creates an automatic compaction configuration for datasource `wikipedia_hour`. This automatic compaction configuration performs compaction on `wikipedia_hour`, resulting in compacted segments that represent a day interval of data.
In this example:
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] Automatic compaction API documentation refactor (druid)
Posted by "ektravel (via GitHub)" <gi...@apache.org>.
ektravel commented on code in PR #14740:
URL: https://github.com/apache/druid/pull/14740#discussion_r1290629845
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,762 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This topic describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. You can configure automatic compaction in the Druid web console or API.
-## Automatic compaction status
+In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments.
+
+## Manage automatic compaction
+
+### Update capacity for compaction tasks
+
+Update the capacity for compaction tasks. The minimum number of compaction tasks is 1 and the maximum is 2147483652.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction/taskslots</code>
+
+#### Query parameters
+
+To limit the maximum number of compaction tasks, use the optional query parameters `ratio` and `max`:
+
+* `ratio` (optional)
+ * Type: Float
+ * Default: 0.1
+ * Limit the ratio of the total task slots to compaction task slots.
+* `max` (optional)
+ * Type: Int
+ * Default: 2147483647
+ * Limit the maximum number of task slots for compaction tasks.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully updated compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Invalid `max` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000"
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000 HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` message code and an empty response body.
+
+### Create or update automatic compaction configuration
+
+Creates or updates the automatic compaction configuration for a datasource. You pass the automatic compaction as a JSON object in the request body.
+
+Automatic compaction configuration require only the `dataSource` property. Druid fills all other properties with default values if not specified. See [Automatic compaction dynamic configuration](../configuration/index.md#automatic-compaction-dynamic-configuration) for configuration details.
+
+Note that this endpoint returns an HTTP `200 OK` message code even if the datasource name does not exist.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully submitted auto compaction configuration*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+#### Sample request
+
+The following example creates an automatic compaction configuration for datasource `wikipedia_hour`. This automatic compaction configuration performs compaction on `wikipedia_hour`, resulting in compacted segments that represent a day interval of data. In this example:
+
+* `wikipedia_hour` is a datasource with `HOUR` segment granularity.
+* `skipOffsetFromLatest` is set to `PT0S`, meaning that no data is skipped.
+* `partitionsSpec` is set to the default `dynamic`, allowing Druid to dynamically determine the optimal partitioning strategy.
+* `type` is set to `index_parallel`, meaning that parallel indexing is used.
+* `segmentGranularity` is set to `DAY`, meaning that each compacted segment is a day of data.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
-`GET /druid/coordinator/v1/compaction/progress?dataSource={dataSource}`
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"\
+--header 'Content-Type: application/json' \
+--data '{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}'
+```
-Returns the total size of segments awaiting compaction for the given dataSource. The specified dataSource must have [automatic compaction](../data-management/automatic-compaction.md) enabled.
+<!--HTTP-->
-`GET /druid/coordinator/v1/compaction/status`
+```HTTP
+POST /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+Content-Type: application/json
+Content-Length: 281
-Returns the status and statistics from the auto-compaction run of all dataSources which have auto-compaction enabled in the latest run. The response payload includes a list of `latestStatus` objects. Each `latestStatus` represents the status for a dataSource (which has/had auto-compaction enabled).
+{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}
+```
-The `latestStatus` object has the following keys:
-* `dataSource`: name of the datasource for this status information
-* `scheduleStatus`: auto-compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active auto-compaction config submitted. Otherwise, returns `NOT_ENABLED`.
-* `bytesAwaitingCompaction`: total bytes of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `bytesCompacted`: total bytes of this datasource that are already compacted with the spec set in the auto-compaction config
-* `bytesSkipped`: total bytes of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `segmentCountAwaitingCompaction`: total number of segments of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `segmentCountCompacted`: total number of segments of this datasource that are already compacted with the spec set in the auto-compaction config
-* `segmentCountSkipped`: total number of segments of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `intervalCountAwaitingCompaction`: total number of intervals of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `intervalCountCompacted`: total number of intervals of this datasource that are already compacted with the spec set in the auto-compaction config
-* `intervalCountSkipped`: total number of intervals of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
+<!--END_DOCUSAURUS_CODE_TABS-->
-`GET /druid/coordinator/v1/compaction/status?dataSource={dataSource}`
+#### Sample response
-Similar to the API `/druid/coordinator/v1/compaction/status` above but filters response to only return information for the dataSource given.
-The dataSource must have auto-compaction enabled.
+A successful request returns an HTTP `200 OK` message code and an empty response body.
+
+
+### Remove automatic compaction configuration
+
+Removes the automatic compaction configuration for a datasource. This updates the compaction status of the datasource to "Not enabled."
+
+#### URL
+
+<code class="deleteAPI">DELETE</code> <code>/druid/coordinator/v1/config/compaction/:dataSource</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully deleted automatic compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Datasource does not have automatic compaction or invalid datasource name*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request DELETE "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+DELETE /druid/coordinator/v1/config/compaction/wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` message code and an empty response body.
## Automatic compaction configuration
-`GET /druid/coordinator/v1/config/compaction`
+### Get all automatic compaction configurations
+
+Retrieves all automatic compaction configurations. Returns a `compactionConfigs` object containing the active automatic compaction configuration of all datasources.
+
+You can use this endpoint to retrieve `compactionTaskSlotRatio` and `maxCompactionTaskSlots` values for managing resource allocation of compaction tasks.
+
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved automatic compaction configurations*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
-Returns all automatic compaction configs.
+```json
+{
+ "compactionConfigs": [
+ {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ {
+ "dataSource": "wikipedia",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ }
+ ],
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+}
+```
+</details>
-`GET /druid/coordinator/v1/config/compaction/{dataSource}`
+### Get automatic compaction configuration
-Returns an automatic compaction config of a dataSource.
+Retrieves the automatic compaction configuration of a datasource.
Review Comment:
```suggestion
Retrieves the automatic compaction configuration for a datasource.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] Automatic compaction API documentation refactor (druid)
Posted by "ektravel (via GitHub)" <gi...@apache.org>.
ektravel commented on code in PR #14740:
URL: https://github.com/apache/druid/pull/14740#discussion_r1290638218
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,762 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This topic describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. You can configure automatic compaction in the Druid web console or API.
-## Automatic compaction status
+In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments.
+
+## Manage automatic compaction
+
+### Update capacity for compaction tasks
+
+Update the capacity for compaction tasks. The minimum number of compaction tasks is 1 and the maximum is 2147483652.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction/taskslots</code>
+
+#### Query parameters
+
+To limit the maximum number of compaction tasks, use the optional query parameters `ratio` and `max`:
+
+* `ratio` (optional)
+ * Type: Float
+ * Default: 0.1
+ * Limit the ratio of the total task slots to compaction task slots.
+* `max` (optional)
+ * Type: Int
+ * Default: 2147483647
+ * Limit the maximum number of task slots for compaction tasks.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully updated compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Invalid `max` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000"
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000 HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` message code and an empty response body.
+
+### Create or update automatic compaction configuration
+
+Creates or updates the automatic compaction configuration for a datasource. You pass the automatic compaction as a JSON object in the request body.
+
+Automatic compaction configuration require only the `dataSource` property. Druid fills all other properties with default values if not specified. See [Automatic compaction dynamic configuration](../configuration/index.md#automatic-compaction-dynamic-configuration) for configuration details.
+
+Note that this endpoint returns an HTTP `200 OK` message code even if the datasource name does not exist.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully submitted auto compaction configuration*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+#### Sample request
+
+The following example creates an automatic compaction configuration for datasource `wikipedia_hour`. This automatic compaction configuration performs compaction on `wikipedia_hour`, resulting in compacted segments that represent a day interval of data. In this example:
+
+* `wikipedia_hour` is a datasource with `HOUR` segment granularity.
+* `skipOffsetFromLatest` is set to `PT0S`, meaning that no data is skipped.
+* `partitionsSpec` is set to the default `dynamic`, allowing Druid to dynamically determine the optimal partitioning strategy.
+* `type` is set to `index_parallel`, meaning that parallel indexing is used.
+* `segmentGranularity` is set to `DAY`, meaning that each compacted segment is a day of data.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
-`GET /druid/coordinator/v1/compaction/progress?dataSource={dataSource}`
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"\
+--header 'Content-Type: application/json' \
+--data '{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}'
+```
-Returns the total size of segments awaiting compaction for the given dataSource. The specified dataSource must have [automatic compaction](../data-management/automatic-compaction.md) enabled.
+<!--HTTP-->
-`GET /druid/coordinator/v1/compaction/status`
+```HTTP
+POST /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+Content-Type: application/json
+Content-Length: 281
-Returns the status and statistics from the auto-compaction run of all dataSources which have auto-compaction enabled in the latest run. The response payload includes a list of `latestStatus` objects. Each `latestStatus` represents the status for a dataSource (which has/had auto-compaction enabled).
+{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}
+```
-The `latestStatus` object has the following keys:
-* `dataSource`: name of the datasource for this status information
-* `scheduleStatus`: auto-compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active auto-compaction config submitted. Otherwise, returns `NOT_ENABLED`.
-* `bytesAwaitingCompaction`: total bytes of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `bytesCompacted`: total bytes of this datasource that are already compacted with the spec set in the auto-compaction config
-* `bytesSkipped`: total bytes of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `segmentCountAwaitingCompaction`: total number of segments of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `segmentCountCompacted`: total number of segments of this datasource that are already compacted with the spec set in the auto-compaction config
-* `segmentCountSkipped`: total number of segments of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `intervalCountAwaitingCompaction`: total number of intervals of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `intervalCountCompacted`: total number of intervals of this datasource that are already compacted with the spec set in the auto-compaction config
-* `intervalCountSkipped`: total number of intervals of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
+<!--END_DOCUSAURUS_CODE_TABS-->
-`GET /druid/coordinator/v1/compaction/status?dataSource={dataSource}`
+#### Sample response
-Similar to the API `/druid/coordinator/v1/compaction/status` above but filters response to only return information for the dataSource given.
-The dataSource must have auto-compaction enabled.
+A successful request returns an HTTP `200 OK` message code and an empty response body.
+
+
+### Remove automatic compaction configuration
+
+Removes the automatic compaction configuration for a datasource. This updates the compaction status of the datasource to "Not enabled."
+
+#### URL
+
+<code class="deleteAPI">DELETE</code> <code>/druid/coordinator/v1/config/compaction/:dataSource</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully deleted automatic compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Datasource does not have automatic compaction or invalid datasource name*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request DELETE "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+DELETE /druid/coordinator/v1/config/compaction/wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` message code and an empty response body.
## Automatic compaction configuration
-`GET /druid/coordinator/v1/config/compaction`
+### Get all automatic compaction configurations
+
+Retrieves all automatic compaction configurations. Returns a `compactionConfigs` object containing the active automatic compaction configuration of all datasources.
+
+You can use this endpoint to retrieve `compactionTaskSlotRatio` and `maxCompactionTaskSlots` values for managing resource allocation of compaction tasks.
+
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved automatic compaction configurations*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
-Returns all automatic compaction configs.
+```json
+{
+ "compactionConfigs": [
+ {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ {
+ "dataSource": "wikipedia",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ }
+ ],
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+}
+```
+</details>
-`GET /druid/coordinator/v1/config/compaction/{dataSource}`
+### Get automatic compaction configuration
-Returns an automatic compaction config of a dataSource.
+Retrieves the automatic compaction configuration of a datasource.
-`GET /druid/coordinator/v1/config/compaction/{dataSource}/history?interval={interval}&count={count}`
+#### URL
-Returns the history of the automatic compaction config for a dataSource. Optionally accepts `interval` and `count`
-query string parameters to filter by interval and limit the number of results respectively. If the dataSource does not
-exist or there is no compaction history for the dataSource, an empty list is returned.
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction/:dataSource</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved configuration for datasource*
+
+<!--404 NOT FOUND-->
+
+*Invalid datasource or datasource does not have automatic compaction enabled*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+The following example retrieves the automatic compaction configuration for datasource `wikipedia_hour`.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction/wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
+
+```json
+{
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+}
+```
+</details>
+
+### Get automatic compaction configuration history
+
+Retrieves the history of the automatic compaction configuration for a datasource. Returns an empty list if the datasource does not exist or there is no compaction history for the datasource.
The response contains a list of objects with the following keys:
-* `globalConfig`: A json object containing automatic compaction config that applies to the entire cluster.
-* `compactionConfig`: A json object containing the automatic compaction config for the datasource.
-* `auditInfo`: A json object that contains information about the change made - like `author`, `comment` and `ip`.
+* `globalConfig`: A JSON object containing automatic compaction configuration that applies to the entire cluster.
+* `compactionConfig`: A JSON object containing the automatic compaction configuration for the datasource.
+* `auditInfo`: A JSON object that contains information about the change made, such as `author`, `comment` or `ip`.
* `auditTime`: The date and time when the change was made.
-`POST /druid/coordinator/v1/config/compaction/taskslots?ratio={someRatio}&max={someMaxSlots}`
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction/:dataSource/history</code>
+
+#### Query parameters
+* `interval` (optional)
+ * Type: ISO-8601
+ * Limit the results within a specified interval. Use `/` as the delimiter for the interval string.
+* `count` (optional)
+ * Type: Int
+ * Limit the number of results.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved configuration history*
+
+<!--400 BAD REQUEST-->
+
+*Invalid `count` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour/history"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction/wikipedia_hour/history HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
+
+```json
+[
+ {
+ "globalConfig": {
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+ },
+ "compactionConfig": {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "P1D",
+ "tuningConfig": null,
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ "auditInfo": {
+ "author": "",
+ "comment": "",
+ "ip": "127.0.0.1"
+ },
+ "auditTime": "2023-07-31T18:15:19.302Z"
+ },
+ {
+ "globalConfig": {
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+ },
+ "compactionConfig": {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ "auditInfo": {
+ "author": "",
+ "comment": "",
+ "ip": "127.0.0.1"
+ },
+ "auditTime": "2023-07-31T18:16:16.362Z"
+ }
+]
+```
+</details>
+
+## Automatic compaction status
+
+### Get segments awaiting compaction
+
+Returns the total size of segments awaiting compaction for a given datasource. The specified datasource must have automatic compaction enabled.
+
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/compaction/progress?dataSource=:dataSource</code>
+
+#### Query parameter
+* `dataSource` (required)
+ * Type: String
+ * Name of the datasource for this status information
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved segment size awaiting compaction*
+
+<!--404 NOT FOUND-->
+
+*Unknown datasource name or datasource does not have automatic compaction enabled*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+The following example retrieves the remaining segments to be compacted for datasource `wikipedia_hour`.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/compaction/progress?dataSource=wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/compaction/progress?dataSource=wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
+
+```json
+{
+ "remainingSegmentSize": 7615837
+}
+```
+</details>
+
+
+### Get compaction status and statistics
+
+Retrieves an array of `latestStatus` objects representing the status and statistics from the latest automatic compaction run for all datasources with automatic compaction enabled.
+
+The `latestStatus` object has the following properties:
+* `dataSource`: Name of the datasource for this status information.
+* `scheduleStatus`: Automatic compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the datasource has an active automatic compaction configuration submitted. Otherwise, returns `NOT_ENABLED`.
+* `bytesAwaitingCompaction`: Total bytes of this datasource waiting to be compacted by the automatic compaction (only consider intervals/segments that are eligible for automatic compaction).
+* `bytesCompacted`: Total bytes of this datasource that are already compacted with the spec set in the automatic compaction config.
+* `bytesSkipped`: Total bytes of this datasource that are skipped (not eligible for automatic compaction) by the automatic compaction.
+* `segmentCountAwaitingCompaction`: Total number of segments of this datasource waiting to be compacted by the automatic compaction (only consider intervals/segments that are eligible for automatic compaction).
+* `segmentCountCompacted`: Total number of segments of this datasource that are already compacted with the spec set in the automatic compaction configuration.
+* `segmentCountSkipped`: Total number of segments of this datasource that are skipped (not eligible for automatic compaction) by the automatic compaction.
+* `intervalCountAwaitingCompaction`: Total number of intervals of this datasource waiting to be compacted by the automatic compaction (only consider intervals/segments that are eligible for automatic compaction).
+* `intervalCountCompacted`: Total number of intervals of this datasource that are already compacted with the spec set in the automatic compaction configuration.
+* `intervalCountSkipped`: Total number of intervals of this datasource that are skipped (not eligible for automatic compaction) by the automatic compaction.
+
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/compaction/status</code>
+
+#### Query parameters
+* `dataSource` (optional)
+ * Type: String
+ * Filter the result by name of specific datasource.
Review Comment:
```suggestion
* Filter the result by name of a specific datasource.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] Automatic compaction API documentation refactor (druid)
Posted by "ektravel (via GitHub)" <gi...@apache.org>.
ektravel commented on code in PR #14740:
URL: https://github.com/apache/druid/pull/14740#discussion_r1290628738
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,762 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This topic describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. You can configure automatic compaction in the Druid web console or API.
-## Automatic compaction status
+In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments.
+
+## Manage automatic compaction
+
+### Update capacity for compaction tasks
+
+Update the capacity for compaction tasks. The minimum number of compaction tasks is 1 and the maximum is 2147483652.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction/taskslots</code>
+
+#### Query parameters
+
+To limit the maximum number of compaction tasks, use the optional query parameters `ratio` and `max`:
+
+* `ratio` (optional)
+ * Type: Float
+ * Default: 0.1
+ * Limit the ratio of the total task slots to compaction task slots.
+* `max` (optional)
+ * Type: Int
+ * Default: 2147483647
+ * Limit the maximum number of task slots for compaction tasks.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully updated compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Invalid `max` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000"
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000 HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` message code and an empty response body.
+
+### Create or update automatic compaction configuration
+
+Creates or updates the automatic compaction configuration for a datasource. You pass the automatic compaction as a JSON object in the request body.
+
+Automatic compaction configuration require only the `dataSource` property. Druid fills all other properties with default values if not specified. See [Automatic compaction dynamic configuration](../configuration/index.md#automatic-compaction-dynamic-configuration) for configuration details.
+
+Note that this endpoint returns an HTTP `200 OK` message code even if the datasource name does not exist.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully submitted auto compaction configuration*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+#### Sample request
+
+The following example creates an automatic compaction configuration for datasource `wikipedia_hour`. This automatic compaction configuration performs compaction on `wikipedia_hour`, resulting in compacted segments that represent a day interval of data. In this example:
+
+* `wikipedia_hour` is a datasource with `HOUR` segment granularity.
+* `skipOffsetFromLatest` is set to `PT0S`, meaning that no data is skipped.
+* `partitionsSpec` is set to the default `dynamic`, allowing Druid to dynamically determine the optimal partitioning strategy.
+* `type` is set to `index_parallel`, meaning that parallel indexing is used.
+* `segmentGranularity` is set to `DAY`, meaning that each compacted segment is a day of data.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
-`GET /druid/coordinator/v1/compaction/progress?dataSource={dataSource}`
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"\
+--header 'Content-Type: application/json' \
+--data '{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}'
+```
-Returns the total size of segments awaiting compaction for the given dataSource. The specified dataSource must have [automatic compaction](../data-management/automatic-compaction.md) enabled.
+<!--HTTP-->
-`GET /druid/coordinator/v1/compaction/status`
+```HTTP
+POST /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+Content-Type: application/json
+Content-Length: 281
-Returns the status and statistics from the auto-compaction run of all dataSources which have auto-compaction enabled in the latest run. The response payload includes a list of `latestStatus` objects. Each `latestStatus` represents the status for a dataSource (which has/had auto-compaction enabled).
+{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}
+```
-The `latestStatus` object has the following keys:
-* `dataSource`: name of the datasource for this status information
-* `scheduleStatus`: auto-compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active auto-compaction config submitted. Otherwise, returns `NOT_ENABLED`.
-* `bytesAwaitingCompaction`: total bytes of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `bytesCompacted`: total bytes of this datasource that are already compacted with the spec set in the auto-compaction config
-* `bytesSkipped`: total bytes of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `segmentCountAwaitingCompaction`: total number of segments of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `segmentCountCompacted`: total number of segments of this datasource that are already compacted with the spec set in the auto-compaction config
-* `segmentCountSkipped`: total number of segments of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `intervalCountAwaitingCompaction`: total number of intervals of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `intervalCountCompacted`: total number of intervals of this datasource that are already compacted with the spec set in the auto-compaction config
-* `intervalCountSkipped`: total number of intervals of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
+<!--END_DOCUSAURUS_CODE_TABS-->
-`GET /druid/coordinator/v1/compaction/status?dataSource={dataSource}`
+#### Sample response
-Similar to the API `/druid/coordinator/v1/compaction/status` above but filters response to only return information for the dataSource given.
-The dataSource must have auto-compaction enabled.
+A successful request returns an HTTP `200 OK` message code and an empty response body.
+
+
+### Remove automatic compaction configuration
+
+Removes the automatic compaction configuration for a datasource. This updates the compaction status of the datasource to "Not enabled."
+
+#### URL
+
+<code class="deleteAPI">DELETE</code> <code>/druid/coordinator/v1/config/compaction/:dataSource</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully deleted automatic compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Datasource does not have automatic compaction or invalid datasource name*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request DELETE "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+DELETE /druid/coordinator/v1/config/compaction/wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` message code and an empty response body.
## Automatic compaction configuration
-`GET /druid/coordinator/v1/config/compaction`
+### Get all automatic compaction configurations
+
+Retrieves all automatic compaction configurations. Returns a `compactionConfigs` object containing the active automatic compaction configuration of all datasources.
Review Comment:
```suggestion
Retrieves all automatic compaction configurations. Returns a `compactionConfigs` object containing the active automatic compaction configurations of all datasources.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] Automatic compaction API documentation refactor (druid)
Posted by "ektravel (via GitHub)" <gi...@apache.org>.
ektravel commented on code in PR #14740:
URL: https://github.com/apache/druid/pull/14740#discussion_r1290615197
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,762 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This topic describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. You can configure automatic compaction in the Druid web console or API.
-## Automatic compaction status
+In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments.
+
+## Manage automatic compaction
+
+### Update capacity for compaction tasks
+
+Update the capacity for compaction tasks. The minimum number of compaction tasks is 1 and the maximum is 2147483652.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction/taskslots</code>
+
+#### Query parameters
+
+To limit the maximum number of compaction tasks, use the optional query parameters `ratio` and `max`:
+
+* `ratio` (optional)
+ * Type: Float
+ * Default: 0.1
+ * Limit the ratio of the total task slots to compaction task slots.
+* `max` (optional)
+ * Type: Int
+ * Default: 2147483647
+ * Limit the maximum number of task slots for compaction tasks.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully updated compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Invalid `max` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000"
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000 HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` message code and an empty response body.
+
+### Create or update automatic compaction configuration
+
+Creates or updates the automatic compaction configuration for a datasource. You pass the automatic compaction as a JSON object in the request body.
+
+Automatic compaction configuration require only the `dataSource` property. Druid fills all other properties with default values if not specified. See [Automatic compaction dynamic configuration](../configuration/index.md#automatic-compaction-dynamic-configuration) for configuration details.
Review Comment:
```suggestion
The automatic compaction configuration requires only the `dataSource` property. Druid fills all other properties with default values if not specified. See [Automatic compaction dynamic configuration](../configuration/index.md#automatic-compaction-dynamic-configuration) for configuration details.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] Automatic compaction API documentation refactor (druid)
Posted by "ektravel (via GitHub)" <gi...@apache.org>.
ektravel commented on code in PR #14740:
URL: https://github.com/apache/druid/pull/14740#discussion_r1290612396
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,762 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This topic describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. You can configure automatic compaction in the Druid web console or API.
-## Automatic compaction status
+In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments.
+
+## Manage automatic compaction
+
+### Update capacity for compaction tasks
+
+Update the capacity for compaction tasks. The minimum number of compaction tasks is 1 and the maximum is 2147483652.
Review Comment:
```suggestion
Updates the capacity for compaction tasks. The minimum number of compaction tasks is 1 and the maximum is 2147483652.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] Automatic compaction API documentation refactor (druid)
Posted by "ektravel (via GitHub)" <gi...@apache.org>.
ektravel commented on code in PR #14740:
URL: https://github.com/apache/druid/pull/14740#discussion_r1290633458
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,762 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This topic describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. You can configure automatic compaction in the Druid web console or API.
-## Automatic compaction status
+In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments.
+
+## Manage automatic compaction
+
+### Update capacity for compaction tasks
+
+Update the capacity for compaction tasks. The minimum number of compaction tasks is 1 and the maximum is 2147483652.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction/taskslots</code>
+
+#### Query parameters
+
+To limit the maximum number of compaction tasks, use the optional query parameters `ratio` and `max`:
+
+* `ratio` (optional)
+ * Type: Float
+ * Default: 0.1
+ * Limit the ratio of the total task slots to compaction task slots.
+* `max` (optional)
+ * Type: Int
+ * Default: 2147483647
+ * Limit the maximum number of task slots for compaction tasks.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully updated compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Invalid `max` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000"
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000 HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` message code and an empty response body.
+
+### Create or update automatic compaction configuration
+
+Creates or updates the automatic compaction configuration for a datasource. You pass the automatic compaction as a JSON object in the request body.
+
+Automatic compaction configuration require only the `dataSource` property. Druid fills all other properties with default values if not specified. See [Automatic compaction dynamic configuration](../configuration/index.md#automatic-compaction-dynamic-configuration) for configuration details.
+
+Note that this endpoint returns an HTTP `200 OK` message code even if the datasource name does not exist.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully submitted auto compaction configuration*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+#### Sample request
+
+The following example creates an automatic compaction configuration for datasource `wikipedia_hour`. This automatic compaction configuration performs compaction on `wikipedia_hour`, resulting in compacted segments that represent a day interval of data. In this example:
+
+* `wikipedia_hour` is a datasource with `HOUR` segment granularity.
+* `skipOffsetFromLatest` is set to `PT0S`, meaning that no data is skipped.
+* `partitionsSpec` is set to the default `dynamic`, allowing Druid to dynamically determine the optimal partitioning strategy.
+* `type` is set to `index_parallel`, meaning that parallel indexing is used.
+* `segmentGranularity` is set to `DAY`, meaning that each compacted segment is a day of data.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
-`GET /druid/coordinator/v1/compaction/progress?dataSource={dataSource}`
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"\
+--header 'Content-Type: application/json' \
+--data '{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}'
+```
-Returns the total size of segments awaiting compaction for the given dataSource. The specified dataSource must have [automatic compaction](../data-management/automatic-compaction.md) enabled.
+<!--HTTP-->
-`GET /druid/coordinator/v1/compaction/status`
+```HTTP
+POST /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+Content-Type: application/json
+Content-Length: 281
-Returns the status and statistics from the auto-compaction run of all dataSources which have auto-compaction enabled in the latest run. The response payload includes a list of `latestStatus` objects. Each `latestStatus` represents the status for a dataSource (which has/had auto-compaction enabled).
+{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}
+```
-The `latestStatus` object has the following keys:
-* `dataSource`: name of the datasource for this status information
-* `scheduleStatus`: auto-compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active auto-compaction config submitted. Otherwise, returns `NOT_ENABLED`.
-* `bytesAwaitingCompaction`: total bytes of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `bytesCompacted`: total bytes of this datasource that are already compacted with the spec set in the auto-compaction config
-* `bytesSkipped`: total bytes of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `segmentCountAwaitingCompaction`: total number of segments of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `segmentCountCompacted`: total number of segments of this datasource that are already compacted with the spec set in the auto-compaction config
-* `segmentCountSkipped`: total number of segments of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `intervalCountAwaitingCompaction`: total number of intervals of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `intervalCountCompacted`: total number of intervals of this datasource that are already compacted with the spec set in the auto-compaction config
-* `intervalCountSkipped`: total number of intervals of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
+<!--END_DOCUSAURUS_CODE_TABS-->
-`GET /druid/coordinator/v1/compaction/status?dataSource={dataSource}`
+#### Sample response
-Similar to the API `/druid/coordinator/v1/compaction/status` above but filters response to only return information for the dataSource given.
-The dataSource must have auto-compaction enabled.
+A successful request returns an HTTP `200 OK` message code and an empty response body.
+
+
+### Remove automatic compaction configuration
+
+Removes the automatic compaction configuration for a datasource. This updates the compaction status of the datasource to "Not enabled."
+
+#### URL
+
+<code class="deleteAPI">DELETE</code> <code>/druid/coordinator/v1/config/compaction/:dataSource</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully deleted automatic compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Datasource does not have automatic compaction or invalid datasource name*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request DELETE "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+DELETE /druid/coordinator/v1/config/compaction/wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` message code and an empty response body.
## Automatic compaction configuration
-`GET /druid/coordinator/v1/config/compaction`
+### Get all automatic compaction configurations
+
+Retrieves all automatic compaction configurations. Returns a `compactionConfigs` object containing the active automatic compaction configuration of all datasources.
+
+You can use this endpoint to retrieve `compactionTaskSlotRatio` and `maxCompactionTaskSlots` values for managing resource allocation of compaction tasks.
+
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved automatic compaction configurations*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
-Returns all automatic compaction configs.
+```json
+{
+ "compactionConfigs": [
+ {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ {
+ "dataSource": "wikipedia",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ }
+ ],
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+}
+```
+</details>
-`GET /druid/coordinator/v1/config/compaction/{dataSource}`
+### Get automatic compaction configuration
-Returns an automatic compaction config of a dataSource.
+Retrieves the automatic compaction configuration of a datasource.
-`GET /druid/coordinator/v1/config/compaction/{dataSource}/history?interval={interval}&count={count}`
+#### URL
-Returns the history of the automatic compaction config for a dataSource. Optionally accepts `interval` and `count`
-query string parameters to filter by interval and limit the number of results respectively. If the dataSource does not
-exist or there is no compaction history for the dataSource, an empty list is returned.
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction/:dataSource</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved configuration for datasource*
+
+<!--404 NOT FOUND-->
+
+*Invalid datasource or datasource does not have automatic compaction enabled*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+The following example retrieves the automatic compaction configuration for datasource `wikipedia_hour`.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction/wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
+
+```json
+{
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+}
+```
+</details>
+
+### Get automatic compaction configuration history
+
+Retrieves the history of the automatic compaction configuration for a datasource. Returns an empty list if the datasource does not exist or there is no compaction history for the datasource.
The response contains a list of objects with the following keys:
-* `globalConfig`: A json object containing automatic compaction config that applies to the entire cluster.
-* `compactionConfig`: A json object containing the automatic compaction config for the datasource.
-* `auditInfo`: A json object that contains information about the change made - like `author`, `comment` and `ip`.
+* `globalConfig`: A JSON object containing automatic compaction configuration that applies to the entire cluster.
+* `compactionConfig`: A JSON object containing the automatic compaction configuration for the datasource.
+* `auditInfo`: A JSON object that contains information about the change made, such as `author`, `comment` or `ip`.
* `auditTime`: The date and time when the change was made.
-`POST /druid/coordinator/v1/config/compaction/taskslots?ratio={someRatio}&max={someMaxSlots}`
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction/:dataSource/history</code>
+
+#### Query parameters
+* `interval` (optional)
+ * Type: ISO-8601
+ * Limit the results within a specified interval. Use `/` as the delimiter for the interval string.
+* `count` (optional)
+ * Type: Int
+ * Limit the number of results.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved configuration history*
+
+<!--400 BAD REQUEST-->
+
+*Invalid `count` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour/history"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction/wikipedia_hour/history HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
+
+```json
+[
+ {
+ "globalConfig": {
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+ },
+ "compactionConfig": {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "P1D",
+ "tuningConfig": null,
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ "auditInfo": {
+ "author": "",
+ "comment": "",
+ "ip": "127.0.0.1"
+ },
+ "auditTime": "2023-07-31T18:15:19.302Z"
+ },
+ {
+ "globalConfig": {
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+ },
+ "compactionConfig": {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ "auditInfo": {
+ "author": "",
+ "comment": "",
+ "ip": "127.0.0.1"
+ },
+ "auditTime": "2023-07-31T18:16:16.362Z"
+ }
+]
+```
+</details>
+
+## Automatic compaction status
+
+### Get segments awaiting compaction
+
+Returns the total size of segments awaiting compaction for a given datasource. The specified datasource must have automatic compaction enabled.
+
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/compaction/progress?dataSource=:dataSource</code>
+
+#### Query parameter
+* `dataSource` (required)
+ * Type: String
+ * Name of the datasource for this status information
Review Comment:
```suggestion
* Name of the datasource for this status information.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] Automatic compaction API documentation refactor (druid)
Posted by "ektravel (via GitHub)" <gi...@apache.org>.
ektravel commented on code in PR #14740:
URL: https://github.com/apache/druid/pull/14740#discussion_r1286357674
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,762 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. Automatic compaction can be configured in the Druid web console or API.
-## Automatic compaction status
+In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments.
-`GET /druid/coordinator/v1/compaction/progress?dataSource={dataSource}`
+## Manage automatic compaction
-Returns the total size of segments awaiting compaction for the given dataSource. The specified dataSource must have [automatic compaction](../data-management/automatic-compaction.md) enabled.
+### Update capacity for compaction tasks
-`GET /druid/coordinator/v1/compaction/status`
+Update the capacity for compaction tasks. Maximum number of compaction tasks is limited by `ratio` and `max`.
+`ratio` is ratio of the total task slots to the compaction task slots and the `max` is the number of task slots for compaction tasks.
-Returns the status and statistics from the auto-compaction run of all dataSources which have auto-compaction enabled in the latest run. The response payload includes a list of `latestStatus` objects. Each `latestStatus` represents the status for a dataSource (which has/had auto-compaction enabled).
+The minimum number of compaction tasks is 1 and the maximum is 2147483652.
-The `latestStatus` object has the following keys:
-* `dataSource`: name of the datasource for this status information
-* `scheduleStatus`: auto-compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active auto-compaction config submitted. Otherwise, returns `NOT_ENABLED`.
-* `bytesAwaitingCompaction`: total bytes of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `bytesCompacted`: total bytes of this datasource that are already compacted with the spec set in the auto-compaction config
-* `bytesSkipped`: total bytes of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `segmentCountAwaitingCompaction`: total number of segments of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `segmentCountCompacted`: total number of segments of this datasource that are already compacted with the spec set in the auto-compaction config
-* `segmentCountSkipped`: total number of segments of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `intervalCountAwaitingCompaction`: total number of intervals of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `intervalCountCompacted`: total number of intervals of this datasource that are already compacted with the spec set in the auto-compaction config
-* `intervalCountSkipped`: total number of intervals of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-
-`GET /druid/coordinator/v1/compaction/status?dataSource={dataSource}`
-
-Similar to the API `/druid/coordinator/v1/compaction/status` above but filters response to only return information for the dataSource given.
-The dataSource must have auto-compaction enabled.
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction/taskslots</code>
+
+#### Query parameters
+* `ratio` (optional)
+ * Type: Float
+ * Default: 0.1
+ * Limit the ratio of the total task slots to compaction task slots.
+* `max` (optional)
+ * Type: Int
+ * Default: 2147483647
+ * Limit the maximum number of task slots for compaction tasks.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully updated compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Invalid `max` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000"
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000 HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
+
+### Create or update datasource automatic compaction configuration
+
+Creates or updates the automatic compaction config for a datasource. The automatic compaction can be submitted as a JSON object in the request body.
+
+Automatic compaction configuration require only the `dataSource` property. All others will be filled with default values if not specified. See [Automatic compaction dynamic configuration](../configuration/index.md#automatic-compaction-dynamic-configuration) for configuration details.
Review Comment:
```suggestion
Automatic compaction configuration require only the `dataSource` property. Druid fills all other properties with default values if not specified. See [Automatic compaction dynamic configuration](../configuration/index.md#automatic-compaction-dynamic-configuration) for configuration details.
```
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,762 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. Automatic compaction can be configured in the Druid web console or API.
-## Automatic compaction status
+In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments.
-`GET /druid/coordinator/v1/compaction/progress?dataSource={dataSource}`
+## Manage automatic compaction
-Returns the total size of segments awaiting compaction for the given dataSource. The specified dataSource must have [automatic compaction](../data-management/automatic-compaction.md) enabled.
+### Update capacity for compaction tasks
-`GET /druid/coordinator/v1/compaction/status`
+Update the capacity for compaction tasks. Maximum number of compaction tasks is limited by `ratio` and `max`.
Review Comment:
```suggestion
Updates the capacity for compaction tasks.
```
Consider moving the information about ratio and max query parameters to the Query parameters section.
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,762 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. Automatic compaction can be configured in the Druid web console or API.
-## Automatic compaction status
+In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments.
-`GET /druid/coordinator/v1/compaction/progress?dataSource={dataSource}`
+## Manage automatic compaction
-Returns the total size of segments awaiting compaction for the given dataSource. The specified dataSource must have [automatic compaction](../data-management/automatic-compaction.md) enabled.
+### Update capacity for compaction tasks
-`GET /druid/coordinator/v1/compaction/status`
+Update the capacity for compaction tasks. Maximum number of compaction tasks is limited by `ratio` and `max`.
+`ratio` is ratio of the total task slots to the compaction task slots and the `max` is the number of task slots for compaction tasks.
-Returns the status and statistics from the auto-compaction run of all dataSources which have auto-compaction enabled in the latest run. The response payload includes a list of `latestStatus` objects. Each `latestStatus` represents the status for a dataSource (which has/had auto-compaction enabled).
+The minimum number of compaction tasks is 1 and the maximum is 2147483652.
-The `latestStatus` object has the following keys:
-* `dataSource`: name of the datasource for this status information
-* `scheduleStatus`: auto-compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active auto-compaction config submitted. Otherwise, returns `NOT_ENABLED`.
-* `bytesAwaitingCompaction`: total bytes of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `bytesCompacted`: total bytes of this datasource that are already compacted with the spec set in the auto-compaction config
-* `bytesSkipped`: total bytes of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `segmentCountAwaitingCompaction`: total number of segments of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `segmentCountCompacted`: total number of segments of this datasource that are already compacted with the spec set in the auto-compaction config
-* `segmentCountSkipped`: total number of segments of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `intervalCountAwaitingCompaction`: total number of intervals of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `intervalCountCompacted`: total number of intervals of this datasource that are already compacted with the spec set in the auto-compaction config
-* `intervalCountSkipped`: total number of intervals of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-
-`GET /druid/coordinator/v1/compaction/status?dataSource={dataSource}`
-
-Similar to the API `/druid/coordinator/v1/compaction/status` above but filters response to only return information for the dataSource given.
-The dataSource must have auto-compaction enabled.
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction/taskslots</code>
+
+#### Query parameters
+* `ratio` (optional)
+ * Type: Float
+ * Default: 0.1
+ * Limit the ratio of the total task slots to compaction task slots.
+* `max` (optional)
+ * Type: Int
+ * Default: 2147483647
+ * Limit the maximum number of task slots for compaction tasks.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully updated compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Invalid `max` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000"
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000 HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
+
+### Create or update datasource automatic compaction configuration
+
+Creates or updates the automatic compaction config for a datasource. The automatic compaction can be submitted as a JSON object in the request body.
Review Comment:
```suggestion
Creates or updates the automatic compaction configuration for a datasource. You pass the automatic compaction as a JSON object in the request body.
```
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,762 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. Automatic compaction can be configured in the Druid web console or API.
-## Automatic compaction status
+In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments.
-`GET /druid/coordinator/v1/compaction/progress?dataSource={dataSource}`
+## Manage automatic compaction
-Returns the total size of segments awaiting compaction for the given dataSource. The specified dataSource must have [automatic compaction](../data-management/automatic-compaction.md) enabled.
+### Update capacity for compaction tasks
-`GET /druid/coordinator/v1/compaction/status`
+Update the capacity for compaction tasks. Maximum number of compaction tasks is limited by `ratio` and `max`.
+`ratio` is ratio of the total task slots to the compaction task slots and the `max` is the number of task slots for compaction tasks.
-Returns the status and statistics from the auto-compaction run of all dataSources which have auto-compaction enabled in the latest run. The response payload includes a list of `latestStatus` objects. Each `latestStatus` represents the status for a dataSource (which has/had auto-compaction enabled).
+The minimum number of compaction tasks is 1 and the maximum is 2147483652.
Review Comment:
I think we should put line 37 right after line 34, before introducing the query parameters.
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,762 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. Automatic compaction can be configured in the Druid web console or API.
-## Automatic compaction status
+In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments.
-`GET /druid/coordinator/v1/compaction/progress?dataSource={dataSource}`
+## Manage automatic compaction
-Returns the total size of segments awaiting compaction for the given dataSource. The specified dataSource must have [automatic compaction](../data-management/automatic-compaction.md) enabled.
+### Update capacity for compaction tasks
-`GET /druid/coordinator/v1/compaction/status`
+Update the capacity for compaction tasks. Maximum number of compaction tasks is limited by `ratio` and `max`.
+`ratio` is ratio of the total task slots to the compaction task slots and the `max` is the number of task slots for compaction tasks.
-Returns the status and statistics from the auto-compaction run of all dataSources which have auto-compaction enabled in the latest run. The response payload includes a list of `latestStatus` objects. Each `latestStatus` represents the status for a dataSource (which has/had auto-compaction enabled).
+The minimum number of compaction tasks is 1 and the maximum is 2147483652.
-The `latestStatus` object has the following keys:
-* `dataSource`: name of the datasource for this status information
-* `scheduleStatus`: auto-compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active auto-compaction config submitted. Otherwise, returns `NOT_ENABLED`.
-* `bytesAwaitingCompaction`: total bytes of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `bytesCompacted`: total bytes of this datasource that are already compacted with the spec set in the auto-compaction config
-* `bytesSkipped`: total bytes of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `segmentCountAwaitingCompaction`: total number of segments of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `segmentCountCompacted`: total number of segments of this datasource that are already compacted with the spec set in the auto-compaction config
-* `segmentCountSkipped`: total number of segments of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `intervalCountAwaitingCompaction`: total number of intervals of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `intervalCountCompacted`: total number of intervals of this datasource that are already compacted with the spec set in the auto-compaction config
-* `intervalCountSkipped`: total number of intervals of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-
-`GET /druid/coordinator/v1/compaction/status?dataSource={dataSource}`
-
-Similar to the API `/druid/coordinator/v1/compaction/status` above but filters response to only return information for the dataSource given.
-The dataSource must have auto-compaction enabled.
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction/taskslots</code>
+
+#### Query parameters
+* `ratio` (optional)
+ * Type: Float
+ * Default: 0.1
+ * Limit the ratio of the total task slots to compaction task slots.
+* `max` (optional)
+ * Type: Int
+ * Default: 2147483647
+ * Limit the maximum number of task slots for compaction tasks.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully updated compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Invalid `max` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000"
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000 HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
+
+### Create or update datasource automatic compaction configuration
+
+Creates or updates the automatic compaction config for a datasource. The automatic compaction can be submitted as a JSON object in the request body.
+
+Automatic compaction configuration require only the `dataSource` property. All others will be filled with default values if not specified. See [Automatic compaction dynamic configuration](../configuration/index.md#automatic-compaction-dynamic-configuration) for configuration details.
+
+Note that this endpoint will return an HTTP `200 OK` even if the datasource name does not exist.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully submitted auto compaction configuration*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+#### Sample request
+
+The following example creates an automatic compaction configuration for datasource `wikipedia_hour`. This automatic compaction configuration performs compaction on `wikipedia_hour`, resulting in compacted segments that represent a day interval of data.
+
+* `wikipedia_hour` is a datasource with `HOUR` segment granularity.
+* `skipOffsetFromLatest` is set to `PT0S`, meaning that no data is skipped.
+* `partitionsSpec` is set to the default `dynamic`, allowing Druid to dynamically determine the optimal partitioning strategy.
+* `type` is set to `index_parallel`, meaning that parallel indexing will be used.
+* `segmentGranularity` is set to `DAY`, meaning that each compacted segment will be a day of data.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"\
+--header 'Content-Type: application/json' \
+--data '{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}'
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+Content-Type: application/json
+Content-Length: 281
+
+{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
Review Comment:
```suggestion
A successful request returns an HTTP `200 OK` message code and an empty response body.
```
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,762 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. Automatic compaction can be configured in the Druid web console or API.
-## Automatic compaction status
+In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments.
-`GET /druid/coordinator/v1/compaction/progress?dataSource={dataSource}`
+## Manage automatic compaction
-Returns the total size of segments awaiting compaction for the given dataSource. The specified dataSource must have [automatic compaction](../data-management/automatic-compaction.md) enabled.
+### Update capacity for compaction tasks
-`GET /druid/coordinator/v1/compaction/status`
+Update the capacity for compaction tasks. Maximum number of compaction tasks is limited by `ratio` and `max`.
+`ratio` is ratio of the total task slots to the compaction task slots and the `max` is the number of task slots for compaction tasks.
-Returns the status and statistics from the auto-compaction run of all dataSources which have auto-compaction enabled in the latest run. The response payload includes a list of `latestStatus` objects. Each `latestStatus` represents the status for a dataSource (which has/had auto-compaction enabled).
+The minimum number of compaction tasks is 1 and the maximum is 2147483652.
-The `latestStatus` object has the following keys:
-* `dataSource`: name of the datasource for this status information
-* `scheduleStatus`: auto-compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active auto-compaction config submitted. Otherwise, returns `NOT_ENABLED`.
-* `bytesAwaitingCompaction`: total bytes of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `bytesCompacted`: total bytes of this datasource that are already compacted with the spec set in the auto-compaction config
-* `bytesSkipped`: total bytes of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `segmentCountAwaitingCompaction`: total number of segments of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `segmentCountCompacted`: total number of segments of this datasource that are already compacted with the spec set in the auto-compaction config
-* `segmentCountSkipped`: total number of segments of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `intervalCountAwaitingCompaction`: total number of intervals of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `intervalCountCompacted`: total number of intervals of this datasource that are already compacted with the spec set in the auto-compaction config
-* `intervalCountSkipped`: total number of intervals of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-
-`GET /druid/coordinator/v1/compaction/status?dataSource={dataSource}`
-
-Similar to the API `/druid/coordinator/v1/compaction/status` above but filters response to only return information for the dataSource given.
-The dataSource must have auto-compaction enabled.
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction/taskslots</code>
+
+#### Query parameters
+* `ratio` (optional)
+ * Type: Float
+ * Default: 0.1
+ * Limit the ratio of the total task slots to compaction task slots.
+* `max` (optional)
+ * Type: Int
+ * Default: 2147483647
+ * Limit the maximum number of task slots for compaction tasks.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully updated compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Invalid `max` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000"
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000 HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
+
+### Create or update datasource automatic compaction configuration
+
+Creates or updates the automatic compaction config for a datasource. The automatic compaction can be submitted as a JSON object in the request body.
+
+Automatic compaction configuration require only the `dataSource` property. All others will be filled with default values if not specified. See [Automatic compaction dynamic configuration](../configuration/index.md#automatic-compaction-dynamic-configuration) for configuration details.
+
+Note that this endpoint will return an HTTP `200 OK` even if the datasource name does not exist.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully submitted auto compaction configuration*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+#### Sample request
+
+The following example creates an automatic compaction configuration for datasource `wikipedia_hour`. This automatic compaction configuration performs compaction on `wikipedia_hour`, resulting in compacted segments that represent a day interval of data.
+
+* `wikipedia_hour` is a datasource with `HOUR` segment granularity.
+* `skipOffsetFromLatest` is set to `PT0S`, meaning that no data is skipped.
+* `partitionsSpec` is set to the default `dynamic`, allowing Druid to dynamically determine the optimal partitioning strategy.
+* `type` is set to `index_parallel`, meaning that parallel indexing will be used.
+* `segmentGranularity` is set to `DAY`, meaning that each compacted segment will be a day of data.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"\
+--header 'Content-Type: application/json' \
+--data '{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}'
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+Content-Type: application/json
+Content-Length: 281
+
+{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
+
+
+### Remove datasource automatic compaction configuration
+
+Removes the automatic compaction configuration for a datasource. This updates the compaction status of the datasource to "Not enabled."
+
+#### URL
+
+<code class="deleteAPI">DELETE</code> <code>/druid/coordinator/v1/config/compaction/:dataSource</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully deleted automatic compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Datasource does not have automatic compaction or invalid datasource name*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request DELETE "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+DELETE /druid/coordinator/v1/config/compaction/wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
## Automatic compaction configuration
-`GET /druid/coordinator/v1/config/compaction`
+### Get all automatic compaction configuration
+
+Retrieves all automatic compaction configurations. Returns a `compactionConfigs` object containing the active automatic compaction configuration of all datasources.
+
+You can use this endpoint to retrieve `compactionTaskSlotRatio` and `maxCompactionTaskSlots` values for managing resource allocation of compaction tasks.
+
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved automatic compaction configurations*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
-Returns all automatic compaction configs.
+```json
+{
+ "compactionConfigs": [
+ {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ {
+ "dataSource": "wikipedia",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ }
+ ],
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+}
+```
+</details>
-`GET /druid/coordinator/v1/config/compaction/{dataSource}`
+### Get datasource automatic compaction configuration
Review Comment:
```suggestion
### Get automatic compaction configuration
```
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,762 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. Automatic compaction can be configured in the Druid web console or API.
-## Automatic compaction status
+In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments.
-`GET /druid/coordinator/v1/compaction/progress?dataSource={dataSource}`
+## Manage automatic compaction
-Returns the total size of segments awaiting compaction for the given dataSource. The specified dataSource must have [automatic compaction](../data-management/automatic-compaction.md) enabled.
+### Update capacity for compaction tasks
-`GET /druid/coordinator/v1/compaction/status`
+Update the capacity for compaction tasks. Maximum number of compaction tasks is limited by `ratio` and `max`.
+`ratio` is ratio of the total task slots to the compaction task slots and the `max` is the number of task slots for compaction tasks.
-Returns the status and statistics from the auto-compaction run of all dataSources which have auto-compaction enabled in the latest run. The response payload includes a list of `latestStatus` objects. Each `latestStatus` represents the status for a dataSource (which has/had auto-compaction enabled).
+The minimum number of compaction tasks is 1 and the maximum is 2147483652.
-The `latestStatus` object has the following keys:
-* `dataSource`: name of the datasource for this status information
-* `scheduleStatus`: auto-compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active auto-compaction config submitted. Otherwise, returns `NOT_ENABLED`.
-* `bytesAwaitingCompaction`: total bytes of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `bytesCompacted`: total bytes of this datasource that are already compacted with the spec set in the auto-compaction config
-* `bytesSkipped`: total bytes of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `segmentCountAwaitingCompaction`: total number of segments of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `segmentCountCompacted`: total number of segments of this datasource that are already compacted with the spec set in the auto-compaction config
-* `segmentCountSkipped`: total number of segments of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `intervalCountAwaitingCompaction`: total number of intervals of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `intervalCountCompacted`: total number of intervals of this datasource that are already compacted with the spec set in the auto-compaction config
-* `intervalCountSkipped`: total number of intervals of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-
-`GET /druid/coordinator/v1/compaction/status?dataSource={dataSource}`
-
-Similar to the API `/druid/coordinator/v1/compaction/status` above but filters response to only return information for the dataSource given.
-The dataSource must have auto-compaction enabled.
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction/taskslots</code>
+
+#### Query parameters
+* `ratio` (optional)
+ * Type: Float
+ * Default: 0.1
+ * Limit the ratio of the total task slots to compaction task slots.
+* `max` (optional)
+ * Type: Int
+ * Default: 2147483647
+ * Limit the maximum number of task slots for compaction tasks.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully updated compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Invalid `max` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000"
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000 HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
+
+### Create or update datasource automatic compaction configuration
+
+Creates or updates the automatic compaction config for a datasource. The automatic compaction can be submitted as a JSON object in the request body.
+
+Automatic compaction configuration require only the `dataSource` property. All others will be filled with default values if not specified. See [Automatic compaction dynamic configuration](../configuration/index.md#automatic-compaction-dynamic-configuration) for configuration details.
+
+Note that this endpoint will return an HTTP `200 OK` even if the datasource name does not exist.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully submitted auto compaction configuration*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+#### Sample request
+
+The following example creates an automatic compaction configuration for datasource `wikipedia_hour`. This automatic compaction configuration performs compaction on `wikipedia_hour`, resulting in compacted segments that represent a day interval of data.
+
+* `wikipedia_hour` is a datasource with `HOUR` segment granularity.
+* `skipOffsetFromLatest` is set to `PT0S`, meaning that no data is skipped.
+* `partitionsSpec` is set to the default `dynamic`, allowing Druid to dynamically determine the optimal partitioning strategy.
+* `type` is set to `index_parallel`, meaning that parallel indexing will be used.
+* `segmentGranularity` is set to `DAY`, meaning that each compacted segment will be a day of data.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"\
+--header 'Content-Type: application/json' \
+--data '{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}'
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+Content-Type: application/json
+Content-Length: 281
+
+{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
+
+
+### Remove datasource automatic compaction configuration
+
+Removes the automatic compaction configuration for a datasource. This updates the compaction status of the datasource to "Not enabled."
+
+#### URL
+
+<code class="deleteAPI">DELETE</code> <code>/druid/coordinator/v1/config/compaction/:dataSource</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully deleted automatic compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Datasource does not have automatic compaction or invalid datasource name*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request DELETE "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+DELETE /druid/coordinator/v1/config/compaction/wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
## Automatic compaction configuration
-`GET /druid/coordinator/v1/config/compaction`
+### Get all automatic compaction configuration
+
+Retrieves all automatic compaction configurations. Returns a `compactionConfigs` object containing the active automatic compaction configuration of all datasources.
+
+You can use this endpoint to retrieve `compactionTaskSlotRatio` and `maxCompactionTaskSlots` values for managing resource allocation of compaction tasks.
+
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved automatic compaction configurations*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
-Returns all automatic compaction configs.
+```json
+{
+ "compactionConfigs": [
+ {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ {
+ "dataSource": "wikipedia",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ }
+ ],
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+}
+```
+</details>
-`GET /druid/coordinator/v1/config/compaction/{dataSource}`
+### Get datasource automatic compaction configuration
-Returns an automatic compaction config of a dataSource.
+Retrieves the automatic compaction configuration of a datasource.
-`GET /druid/coordinator/v1/config/compaction/{dataSource}/history?interval={interval}&count={count}`
+#### URL
-Returns the history of the automatic compaction config for a dataSource. Optionally accepts `interval` and `count`
-query string parameters to filter by interval and limit the number of results respectively. If the dataSource does not
-exist or there is no compaction history for the dataSource, an empty list is returned.
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction/:dataSource</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved configuration for datasource*
+
+<!--404 NOT FOUND-->
+
+*Invalid datasource or datasource does not have automatic compaction enabled*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+The following example retrieves the automatic compaction configuration for datasource `wikipedia_hour`.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction/wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
+
+```json
+{
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+}
+```
+</details>
+
+### Get datasource automatic compaction configuration history
Review Comment:
```suggestion
### Get automatic compaction configuration history
```
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,762 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. Automatic compaction can be configured in the Druid web console or API.
-## Automatic compaction status
+In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments.
-`GET /druid/coordinator/v1/compaction/progress?dataSource={dataSource}`
+## Manage automatic compaction
-Returns the total size of segments awaiting compaction for the given dataSource. The specified dataSource must have [automatic compaction](../data-management/automatic-compaction.md) enabled.
+### Update capacity for compaction tasks
-`GET /druid/coordinator/v1/compaction/status`
+Update the capacity for compaction tasks. Maximum number of compaction tasks is limited by `ratio` and `max`.
+`ratio` is ratio of the total task slots to the compaction task slots and the `max` is the number of task slots for compaction tasks.
-Returns the status and statistics from the auto-compaction run of all dataSources which have auto-compaction enabled in the latest run. The response payload includes a list of `latestStatus` objects. Each `latestStatus` represents the status for a dataSource (which has/had auto-compaction enabled).
+The minimum number of compaction tasks is 1 and the maximum is 2147483652.
-The `latestStatus` object has the following keys:
-* `dataSource`: name of the datasource for this status information
-* `scheduleStatus`: auto-compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active auto-compaction config submitted. Otherwise, returns `NOT_ENABLED`.
-* `bytesAwaitingCompaction`: total bytes of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `bytesCompacted`: total bytes of this datasource that are already compacted with the spec set in the auto-compaction config
-* `bytesSkipped`: total bytes of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `segmentCountAwaitingCompaction`: total number of segments of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `segmentCountCompacted`: total number of segments of this datasource that are already compacted with the spec set in the auto-compaction config
-* `segmentCountSkipped`: total number of segments of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `intervalCountAwaitingCompaction`: total number of intervals of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `intervalCountCompacted`: total number of intervals of this datasource that are already compacted with the spec set in the auto-compaction config
-* `intervalCountSkipped`: total number of intervals of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-
-`GET /druid/coordinator/v1/compaction/status?dataSource={dataSource}`
-
-Similar to the API `/druid/coordinator/v1/compaction/status` above but filters response to only return information for the dataSource given.
-The dataSource must have auto-compaction enabled.
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction/taskslots</code>
+
+#### Query parameters
+* `ratio` (optional)
+ * Type: Float
+ * Default: 0.1
+ * Limit the ratio of the total task slots to compaction task slots.
+* `max` (optional)
+ * Type: Int
+ * Default: 2147483647
+ * Limit the maximum number of task slots for compaction tasks.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully updated compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Invalid `max` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000"
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000 HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
+
+### Create or update datasource automatic compaction configuration
+
+Creates or updates the automatic compaction config for a datasource. The automatic compaction can be submitted as a JSON object in the request body.
+
+Automatic compaction configuration require only the `dataSource` property. All others will be filled with default values if not specified. See [Automatic compaction dynamic configuration](../configuration/index.md#automatic-compaction-dynamic-configuration) for configuration details.
+
+Note that this endpoint will return an HTTP `200 OK` even if the datasource name does not exist.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully submitted auto compaction configuration*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+#### Sample request
+
+The following example creates an automatic compaction configuration for datasource `wikipedia_hour`. This automatic compaction configuration performs compaction on `wikipedia_hour`, resulting in compacted segments that represent a day interval of data.
+
+* `wikipedia_hour` is a datasource with `HOUR` segment granularity.
+* `skipOffsetFromLatest` is set to `PT0S`, meaning that no data is skipped.
+* `partitionsSpec` is set to the default `dynamic`, allowing Druid to dynamically determine the optimal partitioning strategy.
+* `type` is set to `index_parallel`, meaning that parallel indexing will be used.
+* `segmentGranularity` is set to `DAY`, meaning that each compacted segment will be a day of data.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"\
+--header 'Content-Type: application/json' \
+--data '{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}'
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+Content-Type: application/json
+Content-Length: 281
+
+{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
+
+
+### Remove datasource automatic compaction configuration
+
+Removes the automatic compaction configuration for a datasource. This updates the compaction status of the datasource to "Not enabled."
+
+#### URL
+
+<code class="deleteAPI">DELETE</code> <code>/druid/coordinator/v1/config/compaction/:dataSource</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully deleted automatic compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Datasource does not have automatic compaction or invalid datasource name*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request DELETE "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+DELETE /druid/coordinator/v1/config/compaction/wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
## Automatic compaction configuration
-`GET /druid/coordinator/v1/config/compaction`
+### Get all automatic compaction configuration
+
+Retrieves all automatic compaction configurations. Returns a `compactionConfigs` object containing the active automatic compaction configuration of all datasources.
+
+You can use this endpoint to retrieve `compactionTaskSlotRatio` and `maxCompactionTaskSlots` values for managing resource allocation of compaction tasks.
+
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved automatic compaction configurations*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
-Returns all automatic compaction configs.
+```json
+{
+ "compactionConfigs": [
+ {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ {
+ "dataSource": "wikipedia",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ }
+ ],
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+}
+```
+</details>
-`GET /druid/coordinator/v1/config/compaction/{dataSource}`
+### Get datasource automatic compaction configuration
-Returns an automatic compaction config of a dataSource.
+Retrieves the automatic compaction configuration of a datasource.
-`GET /druid/coordinator/v1/config/compaction/{dataSource}/history?interval={interval}&count={count}`
+#### URL
-Returns the history of the automatic compaction config for a dataSource. Optionally accepts `interval` and `count`
-query string parameters to filter by interval and limit the number of results respectively. If the dataSource does not
-exist or there is no compaction history for the dataSource, an empty list is returned.
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction/:dataSource</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved configuration for datasource*
+
+<!--404 NOT FOUND-->
+
+*Invalid datasource or datasource does not have automatic compaction enabled*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+The following example retrieves the automatic compaction configuration for datasource `wikipedia_hour`.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction/wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
+
+```json
+{
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+}
+```
+</details>
+
+### Get datasource automatic compaction configuration history
+
+Retrieves the history of the automatic compaction config for a datasource. Returns an empty list if the datasource does not exist or there is no compaction history for the datasource.
Review Comment:
```suggestion
Retrieves the history of the automatic compaction configuration for a datasource. Returns an empty list if the datasource does not exist or there is no compaction history for the datasource.
```
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,762 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. Automatic compaction can be configured in the Druid web console or API.
-## Automatic compaction status
+In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments.
-`GET /druid/coordinator/v1/compaction/progress?dataSource={dataSource}`
+## Manage automatic compaction
-Returns the total size of segments awaiting compaction for the given dataSource. The specified dataSource must have [automatic compaction](../data-management/automatic-compaction.md) enabled.
+### Update capacity for compaction tasks
-`GET /druid/coordinator/v1/compaction/status`
+Update the capacity for compaction tasks. Maximum number of compaction tasks is limited by `ratio` and `max`.
+`ratio` is ratio of the total task slots to the compaction task slots and the `max` is the number of task slots for compaction tasks.
-Returns the status and statistics from the auto-compaction run of all dataSources which have auto-compaction enabled in the latest run. The response payload includes a list of `latestStatus` objects. Each `latestStatus` represents the status for a dataSource (which has/had auto-compaction enabled).
+The minimum number of compaction tasks is 1 and the maximum is 2147483652.
-The `latestStatus` object has the following keys:
-* `dataSource`: name of the datasource for this status information
-* `scheduleStatus`: auto-compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active auto-compaction config submitted. Otherwise, returns `NOT_ENABLED`.
-* `bytesAwaitingCompaction`: total bytes of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `bytesCompacted`: total bytes of this datasource that are already compacted with the spec set in the auto-compaction config
-* `bytesSkipped`: total bytes of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `segmentCountAwaitingCompaction`: total number of segments of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `segmentCountCompacted`: total number of segments of this datasource that are already compacted with the spec set in the auto-compaction config
-* `segmentCountSkipped`: total number of segments of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `intervalCountAwaitingCompaction`: total number of intervals of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `intervalCountCompacted`: total number of intervals of this datasource that are already compacted with the spec set in the auto-compaction config
-* `intervalCountSkipped`: total number of intervals of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-
-`GET /druid/coordinator/v1/compaction/status?dataSource={dataSource}`
-
-Similar to the API `/druid/coordinator/v1/compaction/status` above but filters response to only return information for the dataSource given.
-The dataSource must have auto-compaction enabled.
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction/taskslots</code>
+
+#### Query parameters
+* `ratio` (optional)
+ * Type: Float
+ * Default: 0.1
+ * Limit the ratio of the total task slots to compaction task slots.
+* `max` (optional)
+ * Type: Int
+ * Default: 2147483647
+ * Limit the maximum number of task slots for compaction tasks.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully updated compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Invalid `max` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000"
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000 HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
+
+### Create or update datasource automatic compaction configuration
+
+Creates or updates the automatic compaction config for a datasource. The automatic compaction can be submitted as a JSON object in the request body.
+
+Automatic compaction configuration require only the `dataSource` property. All others will be filled with default values if not specified. See [Automatic compaction dynamic configuration](../configuration/index.md#automatic-compaction-dynamic-configuration) for configuration details.
+
+Note that this endpoint will return an HTTP `200 OK` even if the datasource name does not exist.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully submitted auto compaction configuration*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+#### Sample request
+
+The following example creates an automatic compaction configuration for datasource `wikipedia_hour`. This automatic compaction configuration performs compaction on `wikipedia_hour`, resulting in compacted segments that represent a day interval of data.
+
+* `wikipedia_hour` is a datasource with `HOUR` segment granularity.
+* `skipOffsetFromLatest` is set to `PT0S`, meaning that no data is skipped.
+* `partitionsSpec` is set to the default `dynamic`, allowing Druid to dynamically determine the optimal partitioning strategy.
+* `type` is set to `index_parallel`, meaning that parallel indexing will be used.
+* `segmentGranularity` is set to `DAY`, meaning that each compacted segment will be a day of data.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"\
+--header 'Content-Type: application/json' \
+--data '{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}'
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+Content-Type: application/json
+Content-Length: 281
+
+{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
+
+
+### Remove datasource automatic compaction configuration
+
+Removes the automatic compaction configuration for a datasource. This updates the compaction status of the datasource to "Not enabled."
+
+#### URL
+
+<code class="deleteAPI">DELETE</code> <code>/druid/coordinator/v1/config/compaction/:dataSource</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully deleted automatic compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Datasource does not have automatic compaction or invalid datasource name*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request DELETE "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+DELETE /druid/coordinator/v1/config/compaction/wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
## Automatic compaction configuration
-`GET /druid/coordinator/v1/config/compaction`
+### Get all automatic compaction configuration
+
+Retrieves all automatic compaction configurations. Returns a `compactionConfigs` object containing the active automatic compaction configuration of all datasources.
+
+You can use this endpoint to retrieve `compactionTaskSlotRatio` and `maxCompactionTaskSlots` values for managing resource allocation of compaction tasks.
+
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved automatic compaction configurations*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
-Returns all automatic compaction configs.
+```json
+{
+ "compactionConfigs": [
+ {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ {
+ "dataSource": "wikipedia",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ }
+ ],
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+}
+```
+</details>
-`GET /druid/coordinator/v1/config/compaction/{dataSource}`
+### Get datasource automatic compaction configuration
-Returns an automatic compaction config of a dataSource.
+Retrieves the automatic compaction configuration of a datasource.
-`GET /druid/coordinator/v1/config/compaction/{dataSource}/history?interval={interval}&count={count}`
+#### URL
-Returns the history of the automatic compaction config for a dataSource. Optionally accepts `interval` and `count`
-query string parameters to filter by interval and limit the number of results respectively. If the dataSource does not
-exist or there is no compaction history for the dataSource, an empty list is returned.
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction/:dataSource</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved configuration for datasource*
+
+<!--404 NOT FOUND-->
+
+*Invalid datasource or datasource does not have automatic compaction enabled*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+The following example retrieves the automatic compaction configuration for datasource `wikipedia_hour`.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction/wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
+
+```json
+{
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+}
+```
+</details>
+
+### Get datasource automatic compaction configuration history
+
+Retrieves the history of the automatic compaction config for a datasource. Returns an empty list if the datasource does not exist or there is no compaction history for the datasource.
The response contains a list of objects with the following keys:
-* `globalConfig`: A json object containing automatic compaction config that applies to the entire cluster.
-* `compactionConfig`: A json object containing the automatic compaction config for the datasource.
-* `auditInfo`: A json object that contains information about the change made - like `author`, `comment` and `ip`.
+* `globalConfig`: A JSON object containing automatic compaction config that applies to the entire cluster.
+* `compactionConfig`: A JSON object containing the automatic compaction config for the datasource.
Review Comment:
```suggestion
* `compactionConfig`: A JSON object containing the automatic compaction configuration for the datasource.
```
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,762 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. Automatic compaction can be configured in the Druid web console or API.
-## Automatic compaction status
+In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments.
-`GET /druid/coordinator/v1/compaction/progress?dataSource={dataSource}`
+## Manage automatic compaction
-Returns the total size of segments awaiting compaction for the given dataSource. The specified dataSource must have [automatic compaction](../data-management/automatic-compaction.md) enabled.
+### Update capacity for compaction tasks
-`GET /druid/coordinator/v1/compaction/status`
+Update the capacity for compaction tasks. Maximum number of compaction tasks is limited by `ratio` and `max`.
+`ratio` is ratio of the total task slots to the compaction task slots and the `max` is the number of task slots for compaction tasks.
-Returns the status and statistics from the auto-compaction run of all dataSources which have auto-compaction enabled in the latest run. The response payload includes a list of `latestStatus` objects. Each `latestStatus` represents the status for a dataSource (which has/had auto-compaction enabled).
+The minimum number of compaction tasks is 1 and the maximum is 2147483652.
-The `latestStatus` object has the following keys:
-* `dataSource`: name of the datasource for this status information
-* `scheduleStatus`: auto-compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active auto-compaction config submitted. Otherwise, returns `NOT_ENABLED`.
-* `bytesAwaitingCompaction`: total bytes of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `bytesCompacted`: total bytes of this datasource that are already compacted with the spec set in the auto-compaction config
-* `bytesSkipped`: total bytes of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `segmentCountAwaitingCompaction`: total number of segments of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `segmentCountCompacted`: total number of segments of this datasource that are already compacted with the spec set in the auto-compaction config
-* `segmentCountSkipped`: total number of segments of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `intervalCountAwaitingCompaction`: total number of intervals of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `intervalCountCompacted`: total number of intervals of this datasource that are already compacted with the spec set in the auto-compaction config
-* `intervalCountSkipped`: total number of intervals of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-
-`GET /druid/coordinator/v1/compaction/status?dataSource={dataSource}`
-
-Similar to the API `/druid/coordinator/v1/compaction/status` above but filters response to only return information for the dataSource given.
-The dataSource must have auto-compaction enabled.
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction/taskslots</code>
+
+#### Query parameters
Review Comment:
```suggestion
#### Query parameters
To limit the maximum number of compaction tasks, use the optional query parameters `ratio` and `max`:
```
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,762 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. Automatic compaction can be configured in the Druid web console or API.
-## Automatic compaction status
+In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments.
-`GET /druid/coordinator/v1/compaction/progress?dataSource={dataSource}`
+## Manage automatic compaction
-Returns the total size of segments awaiting compaction for the given dataSource. The specified dataSource must have [automatic compaction](../data-management/automatic-compaction.md) enabled.
+### Update capacity for compaction tasks
-`GET /druid/coordinator/v1/compaction/status`
+Update the capacity for compaction tasks. Maximum number of compaction tasks is limited by `ratio` and `max`.
+`ratio` is ratio of the total task slots to the compaction task slots and the `max` is the number of task slots for compaction tasks.
-Returns the status and statistics from the auto-compaction run of all dataSources which have auto-compaction enabled in the latest run. The response payload includes a list of `latestStatus` objects. Each `latestStatus` represents the status for a dataSource (which has/had auto-compaction enabled).
+The minimum number of compaction tasks is 1 and the maximum is 2147483652.
-The `latestStatus` object has the following keys:
-* `dataSource`: name of the datasource for this status information
-* `scheduleStatus`: auto-compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active auto-compaction config submitted. Otherwise, returns `NOT_ENABLED`.
-* `bytesAwaitingCompaction`: total bytes of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `bytesCompacted`: total bytes of this datasource that are already compacted with the spec set in the auto-compaction config
-* `bytesSkipped`: total bytes of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `segmentCountAwaitingCompaction`: total number of segments of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `segmentCountCompacted`: total number of segments of this datasource that are already compacted with the spec set in the auto-compaction config
-* `segmentCountSkipped`: total number of segments of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `intervalCountAwaitingCompaction`: total number of intervals of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `intervalCountCompacted`: total number of intervals of this datasource that are already compacted with the spec set in the auto-compaction config
-* `intervalCountSkipped`: total number of intervals of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-
-`GET /druid/coordinator/v1/compaction/status?dataSource={dataSource}`
-
-Similar to the API `/druid/coordinator/v1/compaction/status` above but filters response to only return information for the dataSource given.
-The dataSource must have auto-compaction enabled.
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction/taskslots</code>
+
+#### Query parameters
+* `ratio` (optional)
+ * Type: Float
+ * Default: 0.1
+ * Limit the ratio of the total task slots to compaction task slots.
+* `max` (optional)
+ * Type: Int
+ * Default: 2147483647
+ * Limit the maximum number of task slots for compaction tasks.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully updated compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Invalid `max` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000"
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000 HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
+
+### Create or update datasource automatic compaction configuration
+
+Creates or updates the automatic compaction config for a datasource. The automatic compaction can be submitted as a JSON object in the request body.
+
+Automatic compaction configuration require only the `dataSource` property. All others will be filled with default values if not specified. See [Automatic compaction dynamic configuration](../configuration/index.md#automatic-compaction-dynamic-configuration) for configuration details.
+
+Note that this endpoint will return an HTTP `200 OK` even if the datasource name does not exist.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully submitted auto compaction configuration*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+#### Sample request
+
+The following example creates an automatic compaction configuration for datasource `wikipedia_hour`. This automatic compaction configuration performs compaction on `wikipedia_hour`, resulting in compacted segments that represent a day interval of data.
+
+* `wikipedia_hour` is a datasource with `HOUR` segment granularity.
+* `skipOffsetFromLatest` is set to `PT0S`, meaning that no data is skipped.
+* `partitionsSpec` is set to the default `dynamic`, allowing Druid to dynamically determine the optimal partitioning strategy.
+* `type` is set to `index_parallel`, meaning that parallel indexing will be used.
+* `segmentGranularity` is set to `DAY`, meaning that each compacted segment will be a day of data.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"\
+--header 'Content-Type: application/json' \
+--data '{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}'
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+Content-Type: application/json
+Content-Length: 281
+
+{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
+
+
+### Remove datasource automatic compaction configuration
+
+Removes the automatic compaction configuration for a datasource. This updates the compaction status of the datasource to "Not enabled."
+
+#### URL
+
+<code class="deleteAPI">DELETE</code> <code>/druid/coordinator/v1/config/compaction/:dataSource</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully deleted automatic compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Datasource does not have automatic compaction or invalid datasource name*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request DELETE "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+DELETE /druid/coordinator/v1/config/compaction/wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
## Automatic compaction configuration
-`GET /druid/coordinator/v1/config/compaction`
+### Get all automatic compaction configuration
+
+Retrieves all automatic compaction configurations. Returns a `compactionConfigs` object containing the active automatic compaction configuration of all datasources.
+
+You can use this endpoint to retrieve `compactionTaskSlotRatio` and `maxCompactionTaskSlots` values for managing resource allocation of compaction tasks.
+
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved automatic compaction configurations*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
-Returns all automatic compaction configs.
+```json
+{
+ "compactionConfigs": [
+ {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ {
+ "dataSource": "wikipedia",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ }
+ ],
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+}
+```
+</details>
-`GET /druid/coordinator/v1/config/compaction/{dataSource}`
+### Get datasource automatic compaction configuration
-Returns an automatic compaction config of a dataSource.
+Retrieves the automatic compaction configuration of a datasource.
-`GET /druid/coordinator/v1/config/compaction/{dataSource}/history?interval={interval}&count={count}`
+#### URL
-Returns the history of the automatic compaction config for a dataSource. Optionally accepts `interval` and `count`
-query string parameters to filter by interval and limit the number of results respectively. If the dataSource does not
-exist or there is no compaction history for the dataSource, an empty list is returned.
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction/:dataSource</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved configuration for datasource*
+
+<!--404 NOT FOUND-->
+
+*Invalid datasource or datasource does not have automatic compaction enabled*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+The following example retrieves the automatic compaction configuration for datasource `wikipedia_hour`.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction/wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
+
+```json
+{
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+}
+```
+</details>
+
+### Get datasource automatic compaction configuration history
+
+Retrieves the history of the automatic compaction config for a datasource. Returns an empty list if the datasource does not exist or there is no compaction history for the datasource.
The response contains a list of objects with the following keys:
-* `globalConfig`: A json object containing automatic compaction config that applies to the entire cluster.
-* `compactionConfig`: A json object containing the automatic compaction config for the datasource.
-* `auditInfo`: A json object that contains information about the change made - like `author`, `comment` and `ip`.
+* `globalConfig`: A JSON object containing automatic compaction config that applies to the entire cluster.
+* `compactionConfig`: A JSON object containing the automatic compaction config for the datasource.
+* `auditInfo`: A JSON object that contains information about the change made - like `author`, `comment` and `ip`.
* `auditTime`: The date and time when the change was made.
-`POST /druid/coordinator/v1/config/compaction/taskslots?ratio={someRatio}&max={someMaxSlots}`
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction/:dataSource/history</code>
+
+#### Query parameters
+* `interval` (optional)
+ * Type: ISO-8601
+ * Limit the results within a specified interval. Use `/` as the delimiter for the interval string.
+* `count` (optional)
+ * Type: Int
+ * Limit the number of results.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved configuration history*
+
+<!--400 BAD REQUEST-->
+
+*Invalid `count` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour/history"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction/wikipedia_hour/history HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
+
+```json
+[
+ {
+ "globalConfig": {
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+ },
+ "compactionConfig": {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "P1D",
+ "tuningConfig": null,
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ "auditInfo": {
+ "author": "",
+ "comment": "",
+ "ip": "127.0.0.1"
+ },
+ "auditTime": "2023-07-31T18:15:19.302Z"
+ },
+ {
+ "globalConfig": {
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+ },
+ "compactionConfig": {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ "auditInfo": {
+ "author": "",
+ "comment": "",
+ "ip": "127.0.0.1"
+ },
+ "auditTime": "2023-07-31T18:16:16.362Z"
+ }
+]
+```
+</details>
+
+## Automatic compaction status
+
+### Get segments awaiting compaction
+
+Returns the total size of segments awaiting compaction for the given datasource. The specified datasource must have automatic compaction enabled.
+
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/compaction/progress?dataSource=:dataSource</code>
+
+#### Query parameter
+* `dataSource` (required)
+ * Type: String
+ * Name of the datasource for this status information
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved segment size awaiting compaction*
+
+<!--404 NOT FOUND-->
+
+*Unknown datasource name or datasource does not have automatic compaction enabled*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+The following example retrieves the remaining segments to be compacted for datasource `wikipedia_hour`.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/compaction/progress?dataSource=wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/compaction/progress?dataSource=wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
+
+```json
+{
+ "remainingSegmentSize": 7615837
+}
+```
+</details>
+
+
+### Get compaction status and statistics
+
+Retrieves the status and statistics from the automatic compaction run of all datasources which have automatic compaction enabled in the latest run. Returns a list of `latestStatus` objects. Each `latestStatus` represents the status for a datasource with automatic compaction enabled.
+
+The `latestStatus` object has the following keys:
+* `dataSource`: Name of the datasource for this status information.
+* `scheduleStatus`: Automatic compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active automatic compaction config submitted. Otherwise, returns `NOT_ENABLED`.
+* `bytesAwaitingCompaction`: Total bytes of this datasource waiting to be compacted by the automatic compaction (only consider intervals/segments that are eligible for automatic compaction).
+* `bytesCompacted`: Total bytes of this datasource that are already compacted with the spec set in the automatic compaction config.
Review Comment:
```suggestion
* `bytesCompacted`: Total bytes of this datasource that are already compacted with the spec set in the automatic compaction configuration.
```
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,762 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. Automatic compaction can be configured in the Druid web console or API.
-## Automatic compaction status
+In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments.
-`GET /druid/coordinator/v1/compaction/progress?dataSource={dataSource}`
+## Manage automatic compaction
-Returns the total size of segments awaiting compaction for the given dataSource. The specified dataSource must have [automatic compaction](../data-management/automatic-compaction.md) enabled.
+### Update capacity for compaction tasks
-`GET /druid/coordinator/v1/compaction/status`
+Update the capacity for compaction tasks. Maximum number of compaction tasks is limited by `ratio` and `max`.
+`ratio` is ratio of the total task slots to the compaction task slots and the `max` is the number of task slots for compaction tasks.
-Returns the status and statistics from the auto-compaction run of all dataSources which have auto-compaction enabled in the latest run. The response payload includes a list of `latestStatus` objects. Each `latestStatus` represents the status for a dataSource (which has/had auto-compaction enabled).
+The minimum number of compaction tasks is 1 and the maximum is 2147483652.
-The `latestStatus` object has the following keys:
-* `dataSource`: name of the datasource for this status information
-* `scheduleStatus`: auto-compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active auto-compaction config submitted. Otherwise, returns `NOT_ENABLED`.
-* `bytesAwaitingCompaction`: total bytes of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `bytesCompacted`: total bytes of this datasource that are already compacted with the spec set in the auto-compaction config
-* `bytesSkipped`: total bytes of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `segmentCountAwaitingCompaction`: total number of segments of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `segmentCountCompacted`: total number of segments of this datasource that are already compacted with the spec set in the auto-compaction config
-* `segmentCountSkipped`: total number of segments of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `intervalCountAwaitingCompaction`: total number of intervals of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `intervalCountCompacted`: total number of intervals of this datasource that are already compacted with the spec set in the auto-compaction config
-* `intervalCountSkipped`: total number of intervals of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-
-`GET /druid/coordinator/v1/compaction/status?dataSource={dataSource}`
-
-Similar to the API `/druid/coordinator/v1/compaction/status` above but filters response to only return information for the dataSource given.
-The dataSource must have auto-compaction enabled.
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction/taskslots</code>
+
+#### Query parameters
+* `ratio` (optional)
+ * Type: Float
+ * Default: 0.1
+ * Limit the ratio of the total task slots to compaction task slots.
+* `max` (optional)
+ * Type: Int
+ * Default: 2147483647
+ * Limit the maximum number of task slots for compaction tasks.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully updated compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Invalid `max` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000"
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000 HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
+
+### Create or update datasource automatic compaction configuration
+
+Creates or updates the automatic compaction config for a datasource. The automatic compaction can be submitted as a JSON object in the request body.
+
+Automatic compaction configuration require only the `dataSource` property. All others will be filled with default values if not specified. See [Automatic compaction dynamic configuration](../configuration/index.md#automatic-compaction-dynamic-configuration) for configuration details.
+
+Note that this endpoint will return an HTTP `200 OK` even if the datasource name does not exist.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully submitted auto compaction configuration*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+#### Sample request
+
+The following example creates an automatic compaction configuration for datasource `wikipedia_hour`. This automatic compaction configuration performs compaction on `wikipedia_hour`, resulting in compacted segments that represent a day interval of data.
+
+* `wikipedia_hour` is a datasource with `HOUR` segment granularity.
+* `skipOffsetFromLatest` is set to `PT0S`, meaning that no data is skipped.
+* `partitionsSpec` is set to the default `dynamic`, allowing Druid to dynamically determine the optimal partitioning strategy.
+* `type` is set to `index_parallel`, meaning that parallel indexing will be used.
+* `segmentGranularity` is set to `DAY`, meaning that each compacted segment will be a day of data.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"\
+--header 'Content-Type: application/json' \
+--data '{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}'
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+Content-Type: application/json
+Content-Length: 281
+
+{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
+
+
+### Remove datasource automatic compaction configuration
+
+Removes the automatic compaction configuration for a datasource. This updates the compaction status of the datasource to "Not enabled."
+
+#### URL
+
+<code class="deleteAPI">DELETE</code> <code>/druid/coordinator/v1/config/compaction/:dataSource</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully deleted automatic compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Datasource does not have automatic compaction or invalid datasource name*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request DELETE "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+DELETE /druid/coordinator/v1/config/compaction/wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
## Automatic compaction configuration
-`GET /druid/coordinator/v1/config/compaction`
+### Get all automatic compaction configuration
+
+Retrieves all automatic compaction configurations. Returns a `compactionConfigs` object containing the active automatic compaction configuration of all datasources.
+
+You can use this endpoint to retrieve `compactionTaskSlotRatio` and `maxCompactionTaskSlots` values for managing resource allocation of compaction tasks.
+
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved automatic compaction configurations*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
-Returns all automatic compaction configs.
+```json
+{
+ "compactionConfigs": [
+ {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ {
+ "dataSource": "wikipedia",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ }
+ ],
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+}
+```
+</details>
-`GET /druid/coordinator/v1/config/compaction/{dataSource}`
+### Get datasource automatic compaction configuration
-Returns an automatic compaction config of a dataSource.
+Retrieves the automatic compaction configuration of a datasource.
-`GET /druid/coordinator/v1/config/compaction/{dataSource}/history?interval={interval}&count={count}`
+#### URL
-Returns the history of the automatic compaction config for a dataSource. Optionally accepts `interval` and `count`
-query string parameters to filter by interval and limit the number of results respectively. If the dataSource does not
-exist or there is no compaction history for the dataSource, an empty list is returned.
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction/:dataSource</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved configuration for datasource*
+
+<!--404 NOT FOUND-->
+
+*Invalid datasource or datasource does not have automatic compaction enabled*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+The following example retrieves the automatic compaction configuration for datasource `wikipedia_hour`.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction/wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
+
+```json
+{
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+}
+```
+</details>
+
+### Get datasource automatic compaction configuration history
+
+Retrieves the history of the automatic compaction config for a datasource. Returns an empty list if the datasource does not exist or there is no compaction history for the datasource.
The response contains a list of objects with the following keys:
-* `globalConfig`: A json object containing automatic compaction config that applies to the entire cluster.
-* `compactionConfig`: A json object containing the automatic compaction config for the datasource.
-* `auditInfo`: A json object that contains information about the change made - like `author`, `comment` and `ip`.
+* `globalConfig`: A JSON object containing automatic compaction config that applies to the entire cluster.
+* `compactionConfig`: A JSON object containing the automatic compaction config for the datasource.
+* `auditInfo`: A JSON object that contains information about the change made - like `author`, `comment` and `ip`.
* `auditTime`: The date and time when the change was made.
-`POST /druid/coordinator/v1/config/compaction/taskslots?ratio={someRatio}&max={someMaxSlots}`
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction/:dataSource/history</code>
+
+#### Query parameters
+* `interval` (optional)
+ * Type: ISO-8601
+ * Limit the results within a specified interval. Use `/` as the delimiter for the interval string.
+* `count` (optional)
+ * Type: Int
+ * Limit the number of results.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved configuration history*
+
+<!--400 BAD REQUEST-->
+
+*Invalid `count` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour/history"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction/wikipedia_hour/history HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
+
+```json
+[
+ {
+ "globalConfig": {
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+ },
+ "compactionConfig": {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "P1D",
+ "tuningConfig": null,
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ "auditInfo": {
+ "author": "",
+ "comment": "",
+ "ip": "127.0.0.1"
+ },
+ "auditTime": "2023-07-31T18:15:19.302Z"
+ },
+ {
+ "globalConfig": {
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+ },
+ "compactionConfig": {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ "auditInfo": {
+ "author": "",
+ "comment": "",
+ "ip": "127.0.0.1"
+ },
+ "auditTime": "2023-07-31T18:16:16.362Z"
+ }
+]
+```
+</details>
+
+## Automatic compaction status
+
+### Get segments awaiting compaction
+
+Returns the total size of segments awaiting compaction for the given datasource. The specified datasource must have automatic compaction enabled.
+
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/compaction/progress?dataSource=:dataSource</code>
+
+#### Query parameter
+* `dataSource` (required)
+ * Type: String
+ * Name of the datasource for this status information
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved segment size awaiting compaction*
+
+<!--404 NOT FOUND-->
+
+*Unknown datasource name or datasource does not have automatic compaction enabled*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+The following example retrieves the remaining segments to be compacted for datasource `wikipedia_hour`.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/compaction/progress?dataSource=wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/compaction/progress?dataSource=wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
+
+```json
+{
+ "remainingSegmentSize": 7615837
+}
+```
+</details>
+
+
+### Get compaction status and statistics
+
+Retrieves the status and statistics from the automatic compaction run of all datasources which have automatic compaction enabled in the latest run. Returns a list of `latestStatus` objects. Each `latestStatus` represents the status for a datasource with automatic compaction enabled.
+
+The `latestStatus` object has the following keys:
+* `dataSource`: Name of the datasource for this status information.
+* `scheduleStatus`: Automatic compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active automatic compaction config submitted. Otherwise, returns `NOT_ENABLED`.
+* `bytesAwaitingCompaction`: Total bytes of this datasource waiting to be compacted by the automatic compaction (only consider intervals/segments that are eligible for automatic compaction).
+* `bytesCompacted`: Total bytes of this datasource that are already compacted with the spec set in the automatic compaction config.
+* `bytesSkipped`: Total bytes of this datasource that are skipped (not eligible for automatic compaction) by the automatic compaction.
+* `segmentCountAwaitingCompaction`: Total number of segments of this datasource waiting to be compacted by the automatic compaction (only consider intervals/segments that are eligible for automatic compaction).
+* `segmentCountCompacted`: Total number of segments of this datasource that are already compacted with the spec set in the automatic compaction config.
Review Comment:
```suggestion
* `segmentCountCompacted`: Total number of segments of this datasource that are already compacted with the spec set in the automatic compaction configuration.
```
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,762 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. Automatic compaction can be configured in the Druid web console or API.
Review Comment:
```suggestion
This topic describes the status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. You can configure automatic compaction in the Druid web console or API.
```
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,762 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. Automatic compaction can be configured in the Druid web console or API.
-## Automatic compaction status
+In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments.
-`GET /druid/coordinator/v1/compaction/progress?dataSource={dataSource}`
+## Manage automatic compaction
-Returns the total size of segments awaiting compaction for the given dataSource. The specified dataSource must have [automatic compaction](../data-management/automatic-compaction.md) enabled.
+### Update capacity for compaction tasks
-`GET /druid/coordinator/v1/compaction/status`
+Update the capacity for compaction tasks. Maximum number of compaction tasks is limited by `ratio` and `max`.
+`ratio` is ratio of the total task slots to the compaction task slots and the `max` is the number of task slots for compaction tasks.
-Returns the status and statistics from the auto-compaction run of all dataSources which have auto-compaction enabled in the latest run. The response payload includes a list of `latestStatus` objects. Each `latestStatus` represents the status for a dataSource (which has/had auto-compaction enabled).
+The minimum number of compaction tasks is 1 and the maximum is 2147483652.
-The `latestStatus` object has the following keys:
-* `dataSource`: name of the datasource for this status information
-* `scheduleStatus`: auto-compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active auto-compaction config submitted. Otherwise, returns `NOT_ENABLED`.
-* `bytesAwaitingCompaction`: total bytes of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `bytesCompacted`: total bytes of this datasource that are already compacted with the spec set in the auto-compaction config
-* `bytesSkipped`: total bytes of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `segmentCountAwaitingCompaction`: total number of segments of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `segmentCountCompacted`: total number of segments of this datasource that are already compacted with the spec set in the auto-compaction config
-* `segmentCountSkipped`: total number of segments of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `intervalCountAwaitingCompaction`: total number of intervals of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `intervalCountCompacted`: total number of intervals of this datasource that are already compacted with the spec set in the auto-compaction config
-* `intervalCountSkipped`: total number of intervals of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-
-`GET /druid/coordinator/v1/compaction/status?dataSource={dataSource}`
-
-Similar to the API `/druid/coordinator/v1/compaction/status` above but filters response to only return information for the dataSource given.
-The dataSource must have auto-compaction enabled.
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction/taskslots</code>
+
+#### Query parameters
+* `ratio` (optional)
+ * Type: Float
+ * Default: 0.1
+ * Limit the ratio of the total task slots to compaction task slots.
+* `max` (optional)
+ * Type: Int
+ * Default: 2147483647
+ * Limit the maximum number of task slots for compaction tasks.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully updated compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Invalid `max` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000"
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000 HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
+
+### Create or update datasource automatic compaction configuration
+
+Creates or updates the automatic compaction config for a datasource. The automatic compaction can be submitted as a JSON object in the request body.
+
+Automatic compaction configuration require only the `dataSource` property. All others will be filled with default values if not specified. See [Automatic compaction dynamic configuration](../configuration/index.md#automatic-compaction-dynamic-configuration) for configuration details.
+
+Note that this endpoint will return an HTTP `200 OK` even if the datasource name does not exist.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully submitted auto compaction configuration*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+#### Sample request
+
+The following example creates an automatic compaction configuration for datasource `wikipedia_hour`. This automatic compaction configuration performs compaction on `wikipedia_hour`, resulting in compacted segments that represent a day interval of data.
+
+* `wikipedia_hour` is a datasource with `HOUR` segment granularity.
+* `skipOffsetFromLatest` is set to `PT0S`, meaning that no data is skipped.
+* `partitionsSpec` is set to the default `dynamic`, allowing Druid to dynamically determine the optimal partitioning strategy.
+* `type` is set to `index_parallel`, meaning that parallel indexing will be used.
+* `segmentGranularity` is set to `DAY`, meaning that each compacted segment will be a day of data.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"\
+--header 'Content-Type: application/json' \
+--data '{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}'
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+Content-Type: application/json
+Content-Length: 281
+
+{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
+
+
+### Remove datasource automatic compaction configuration
Review Comment:
```suggestion
### Remove automatic compaction configuration
```
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,762 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. Automatic compaction can be configured in the Druid web console or API.
-## Automatic compaction status
+In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments.
-`GET /druid/coordinator/v1/compaction/progress?dataSource={dataSource}`
+## Manage automatic compaction
-Returns the total size of segments awaiting compaction for the given dataSource. The specified dataSource must have [automatic compaction](../data-management/automatic-compaction.md) enabled.
+### Update capacity for compaction tasks
-`GET /druid/coordinator/v1/compaction/status`
+Update the capacity for compaction tasks. Maximum number of compaction tasks is limited by `ratio` and `max`.
+`ratio` is ratio of the total task slots to the compaction task slots and the `max` is the number of task slots for compaction tasks.
-Returns the status and statistics from the auto-compaction run of all dataSources which have auto-compaction enabled in the latest run. The response payload includes a list of `latestStatus` objects. Each `latestStatus` represents the status for a dataSource (which has/had auto-compaction enabled).
+The minimum number of compaction tasks is 1 and the maximum is 2147483652.
-The `latestStatus` object has the following keys:
-* `dataSource`: name of the datasource for this status information
-* `scheduleStatus`: auto-compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active auto-compaction config submitted. Otherwise, returns `NOT_ENABLED`.
-* `bytesAwaitingCompaction`: total bytes of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `bytesCompacted`: total bytes of this datasource that are already compacted with the spec set in the auto-compaction config
-* `bytesSkipped`: total bytes of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `segmentCountAwaitingCompaction`: total number of segments of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `segmentCountCompacted`: total number of segments of this datasource that are already compacted with the spec set in the auto-compaction config
-* `segmentCountSkipped`: total number of segments of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `intervalCountAwaitingCompaction`: total number of intervals of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `intervalCountCompacted`: total number of intervals of this datasource that are already compacted with the spec set in the auto-compaction config
-* `intervalCountSkipped`: total number of intervals of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-
-`GET /druid/coordinator/v1/compaction/status?dataSource={dataSource}`
-
-Similar to the API `/druid/coordinator/v1/compaction/status` above but filters response to only return information for the dataSource given.
-The dataSource must have auto-compaction enabled.
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction/taskslots</code>
+
+#### Query parameters
+* `ratio` (optional)
+ * Type: Float
+ * Default: 0.1
+ * Limit the ratio of the total task slots to compaction task slots.
+* `max` (optional)
+ * Type: Int
+ * Default: 2147483647
+ * Limit the maximum number of task slots for compaction tasks.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully updated compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Invalid `max` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000"
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000 HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
+
+### Create or update datasource automatic compaction configuration
+
+Creates or updates the automatic compaction config for a datasource. The automatic compaction can be submitted as a JSON object in the request body.
+
+Automatic compaction configuration require only the `dataSource` property. All others will be filled with default values if not specified. See [Automatic compaction dynamic configuration](../configuration/index.md#automatic-compaction-dynamic-configuration) for configuration details.
+
+Note that this endpoint will return an HTTP `200 OK` even if the datasource name does not exist.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully submitted auto compaction configuration*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+#### Sample request
+
+The following example creates an automatic compaction configuration for datasource `wikipedia_hour`. This automatic compaction configuration performs compaction on `wikipedia_hour`, resulting in compacted segments that represent a day interval of data.
+
+* `wikipedia_hour` is a datasource with `HOUR` segment granularity.
+* `skipOffsetFromLatest` is set to `PT0S`, meaning that no data is skipped.
+* `partitionsSpec` is set to the default `dynamic`, allowing Druid to dynamically determine the optimal partitioning strategy.
+* `type` is set to `index_parallel`, meaning that parallel indexing will be used.
+* `segmentGranularity` is set to `DAY`, meaning that each compacted segment will be a day of data.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"\
+--header 'Content-Type: application/json' \
+--data '{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}'
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+Content-Type: application/json
+Content-Length: 281
+
+{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
+
+
+### Remove datasource automatic compaction configuration
+
+Removes the automatic compaction configuration for a datasource. This updates the compaction status of the datasource to "Not enabled."
+
+#### URL
+
+<code class="deleteAPI">DELETE</code> <code>/druid/coordinator/v1/config/compaction/:dataSource</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully deleted automatic compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Datasource does not have automatic compaction or invalid datasource name*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request DELETE "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+DELETE /druid/coordinator/v1/config/compaction/wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
Review Comment:
```suggestion
A successful request returns an HTTP `200 OK` message code and an empty response body.
```
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,762 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. Automatic compaction can be configured in the Druid web console or API.
-## Automatic compaction status
+In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments.
-`GET /druid/coordinator/v1/compaction/progress?dataSource={dataSource}`
+## Manage automatic compaction
-Returns the total size of segments awaiting compaction for the given dataSource. The specified dataSource must have [automatic compaction](../data-management/automatic-compaction.md) enabled.
+### Update capacity for compaction tasks
-`GET /druid/coordinator/v1/compaction/status`
+Update the capacity for compaction tasks. Maximum number of compaction tasks is limited by `ratio` and `max`.
+`ratio` is ratio of the total task slots to the compaction task slots and the `max` is the number of task slots for compaction tasks.
-Returns the status and statistics from the auto-compaction run of all dataSources which have auto-compaction enabled in the latest run. The response payload includes a list of `latestStatus` objects. Each `latestStatus` represents the status for a dataSource (which has/had auto-compaction enabled).
+The minimum number of compaction tasks is 1 and the maximum is 2147483652.
-The `latestStatus` object has the following keys:
-* `dataSource`: name of the datasource for this status information
-* `scheduleStatus`: auto-compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active auto-compaction config submitted. Otherwise, returns `NOT_ENABLED`.
-* `bytesAwaitingCompaction`: total bytes of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `bytesCompacted`: total bytes of this datasource that are already compacted with the spec set in the auto-compaction config
-* `bytesSkipped`: total bytes of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `segmentCountAwaitingCompaction`: total number of segments of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `segmentCountCompacted`: total number of segments of this datasource that are already compacted with the spec set in the auto-compaction config
-* `segmentCountSkipped`: total number of segments of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `intervalCountAwaitingCompaction`: total number of intervals of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `intervalCountCompacted`: total number of intervals of this datasource that are already compacted with the spec set in the auto-compaction config
-* `intervalCountSkipped`: total number of intervals of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-
-`GET /druid/coordinator/v1/compaction/status?dataSource={dataSource}`
-
-Similar to the API `/druid/coordinator/v1/compaction/status` above but filters response to only return information for the dataSource given.
-The dataSource must have auto-compaction enabled.
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction/taskslots</code>
+
+#### Query parameters
+* `ratio` (optional)
+ * Type: Float
+ * Default: 0.1
+ * Limit the ratio of the total task slots to compaction task slots.
+* `max` (optional)
+ * Type: Int
+ * Default: 2147483647
+ * Limit the maximum number of task slots for compaction tasks.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully updated compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Invalid `max` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000"
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000 HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
+
+### Create or update datasource automatic compaction configuration
+
+Creates or updates the automatic compaction config for a datasource. The automatic compaction can be submitted as a JSON object in the request body.
+
+Automatic compaction configuration require only the `dataSource` property. All others will be filled with default values if not specified. See [Automatic compaction dynamic configuration](../configuration/index.md#automatic-compaction-dynamic-configuration) for configuration details.
+
+Note that this endpoint will return an HTTP `200 OK` even if the datasource name does not exist.
Review Comment:
```suggestion
Note that this endpoint returns an HTTP `200 OK` code message even if the datasource name does not exist.
```
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,762 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. Automatic compaction can be configured in the Druid web console or API.
-## Automatic compaction status
+In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments.
-`GET /druid/coordinator/v1/compaction/progress?dataSource={dataSource}`
+## Manage automatic compaction
-Returns the total size of segments awaiting compaction for the given dataSource. The specified dataSource must have [automatic compaction](../data-management/automatic-compaction.md) enabled.
+### Update capacity for compaction tasks
-`GET /druid/coordinator/v1/compaction/status`
+Update the capacity for compaction tasks. Maximum number of compaction tasks is limited by `ratio` and `max`.
+`ratio` is ratio of the total task slots to the compaction task slots and the `max` is the number of task slots for compaction tasks.
-Returns the status and statistics from the auto-compaction run of all dataSources which have auto-compaction enabled in the latest run. The response payload includes a list of `latestStatus` objects. Each `latestStatus` represents the status for a dataSource (which has/had auto-compaction enabled).
+The minimum number of compaction tasks is 1 and the maximum is 2147483652.
-The `latestStatus` object has the following keys:
-* `dataSource`: name of the datasource for this status information
-* `scheduleStatus`: auto-compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active auto-compaction config submitted. Otherwise, returns `NOT_ENABLED`.
-* `bytesAwaitingCompaction`: total bytes of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `bytesCompacted`: total bytes of this datasource that are already compacted with the spec set in the auto-compaction config
-* `bytesSkipped`: total bytes of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `segmentCountAwaitingCompaction`: total number of segments of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `segmentCountCompacted`: total number of segments of this datasource that are already compacted with the spec set in the auto-compaction config
-* `segmentCountSkipped`: total number of segments of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `intervalCountAwaitingCompaction`: total number of intervals of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `intervalCountCompacted`: total number of intervals of this datasource that are already compacted with the spec set in the auto-compaction config
-* `intervalCountSkipped`: total number of intervals of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-
-`GET /druid/coordinator/v1/compaction/status?dataSource={dataSource}`
-
-Similar to the API `/druid/coordinator/v1/compaction/status` above but filters response to only return information for the dataSource given.
-The dataSource must have auto-compaction enabled.
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction/taskslots</code>
+
+#### Query parameters
+* `ratio` (optional)
+ * Type: Float
+ * Default: 0.1
+ * Limit the ratio of the total task slots to compaction task slots.
+* `max` (optional)
+ * Type: Int
+ * Default: 2147483647
+ * Limit the maximum number of task slots for compaction tasks.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully updated compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Invalid `max` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000"
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000 HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
+
+### Create or update datasource automatic compaction configuration
Review Comment:
```suggestion
### Create or update automatic compaction configuration
```
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,762 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. Automatic compaction can be configured in the Druid web console or API.
-## Automatic compaction status
+In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments.
-`GET /druid/coordinator/v1/compaction/progress?dataSource={dataSource}`
+## Manage automatic compaction
-Returns the total size of segments awaiting compaction for the given dataSource. The specified dataSource must have [automatic compaction](../data-management/automatic-compaction.md) enabled.
+### Update capacity for compaction tasks
-`GET /druid/coordinator/v1/compaction/status`
+Update the capacity for compaction tasks. Maximum number of compaction tasks is limited by `ratio` and `max`.
+`ratio` is ratio of the total task slots to the compaction task slots and the `max` is the number of task slots for compaction tasks.
-Returns the status and statistics from the auto-compaction run of all dataSources which have auto-compaction enabled in the latest run. The response payload includes a list of `latestStatus` objects. Each `latestStatus` represents the status for a dataSource (which has/had auto-compaction enabled).
+The minimum number of compaction tasks is 1 and the maximum is 2147483652.
-The `latestStatus` object has the following keys:
-* `dataSource`: name of the datasource for this status information
-* `scheduleStatus`: auto-compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active auto-compaction config submitted. Otherwise, returns `NOT_ENABLED`.
-* `bytesAwaitingCompaction`: total bytes of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `bytesCompacted`: total bytes of this datasource that are already compacted with the spec set in the auto-compaction config
-* `bytesSkipped`: total bytes of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `segmentCountAwaitingCompaction`: total number of segments of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `segmentCountCompacted`: total number of segments of this datasource that are already compacted with the spec set in the auto-compaction config
-* `segmentCountSkipped`: total number of segments of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `intervalCountAwaitingCompaction`: total number of intervals of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `intervalCountCompacted`: total number of intervals of this datasource that are already compacted with the spec set in the auto-compaction config
-* `intervalCountSkipped`: total number of intervals of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-
-`GET /druid/coordinator/v1/compaction/status?dataSource={dataSource}`
-
-Similar to the API `/druid/coordinator/v1/compaction/status` above but filters response to only return information for the dataSource given.
-The dataSource must have auto-compaction enabled.
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction/taskslots</code>
+
+#### Query parameters
+* `ratio` (optional)
+ * Type: Float
+ * Default: 0.1
+ * Limit the ratio of the total task slots to compaction task slots.
+* `max` (optional)
+ * Type: Int
+ * Default: 2147483647
+ * Limit the maximum number of task slots for compaction tasks.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully updated compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Invalid `max` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000"
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000 HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
+
+### Create or update datasource automatic compaction configuration
+
+Creates or updates the automatic compaction config for a datasource. The automatic compaction can be submitted as a JSON object in the request body.
+
+Automatic compaction configuration require only the `dataSource` property. All others will be filled with default values if not specified. See [Automatic compaction dynamic configuration](../configuration/index.md#automatic-compaction-dynamic-configuration) for configuration details.
+
+Note that this endpoint will return an HTTP `200 OK` even if the datasource name does not exist.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully submitted auto compaction configuration*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+#### Sample request
+
+The following example creates an automatic compaction configuration for datasource `wikipedia_hour`. This automatic compaction configuration performs compaction on `wikipedia_hour`, resulting in compacted segments that represent a day interval of data.
+
Review Comment:
Consider adding an introductory sentence for this list. It can be something as simple as "In this example: "
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,762 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. Automatic compaction can be configured in the Druid web console or API.
-## Automatic compaction status
+In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments.
-`GET /druid/coordinator/v1/compaction/progress?dataSource={dataSource}`
+## Manage automatic compaction
-Returns the total size of segments awaiting compaction for the given dataSource. The specified dataSource must have [automatic compaction](../data-management/automatic-compaction.md) enabled.
+### Update capacity for compaction tasks
-`GET /druid/coordinator/v1/compaction/status`
+Update the capacity for compaction tasks. Maximum number of compaction tasks is limited by `ratio` and `max`.
+`ratio` is ratio of the total task slots to the compaction task slots and the `max` is the number of task slots for compaction tasks.
-Returns the status and statistics from the auto-compaction run of all dataSources which have auto-compaction enabled in the latest run. The response payload includes a list of `latestStatus` objects. Each `latestStatus` represents the status for a dataSource (which has/had auto-compaction enabled).
+The minimum number of compaction tasks is 1 and the maximum is 2147483652.
-The `latestStatus` object has the following keys:
-* `dataSource`: name of the datasource for this status information
-* `scheduleStatus`: auto-compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active auto-compaction config submitted. Otherwise, returns `NOT_ENABLED`.
-* `bytesAwaitingCompaction`: total bytes of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `bytesCompacted`: total bytes of this datasource that are already compacted with the spec set in the auto-compaction config
-* `bytesSkipped`: total bytes of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `segmentCountAwaitingCompaction`: total number of segments of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `segmentCountCompacted`: total number of segments of this datasource that are already compacted with the spec set in the auto-compaction config
-* `segmentCountSkipped`: total number of segments of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `intervalCountAwaitingCompaction`: total number of intervals of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `intervalCountCompacted`: total number of intervals of this datasource that are already compacted with the spec set in the auto-compaction config
-* `intervalCountSkipped`: total number of intervals of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-
-`GET /druid/coordinator/v1/compaction/status?dataSource={dataSource}`
-
-Similar to the API `/druid/coordinator/v1/compaction/status` above but filters response to only return information for the dataSource given.
-The dataSource must have auto-compaction enabled.
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction/taskslots</code>
+
+#### Query parameters
+* `ratio` (optional)
+ * Type: Float
+ * Default: 0.1
+ * Limit the ratio of the total task slots to compaction task slots.
+* `max` (optional)
+ * Type: Int
+ * Default: 2147483647
+ * Limit the maximum number of task slots for compaction tasks.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully updated compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Invalid `max` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000"
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000 HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
+
+### Create or update datasource automatic compaction configuration
+
+Creates or updates the automatic compaction config for a datasource. The automatic compaction can be submitted as a JSON object in the request body.
+
+Automatic compaction configuration require only the `dataSource` property. All others will be filled with default values if not specified. See [Automatic compaction dynamic configuration](../configuration/index.md#automatic-compaction-dynamic-configuration) for configuration details.
+
+Note that this endpoint will return an HTTP `200 OK` even if the datasource name does not exist.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully submitted auto compaction configuration*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+#### Sample request
+
+The following example creates an automatic compaction configuration for datasource `wikipedia_hour`. This automatic compaction configuration performs compaction on `wikipedia_hour`, resulting in compacted segments that represent a day interval of data.
+
+* `wikipedia_hour` is a datasource with `HOUR` segment granularity.
+* `skipOffsetFromLatest` is set to `PT0S`, meaning that no data is skipped.
+* `partitionsSpec` is set to the default `dynamic`, allowing Druid to dynamically determine the optimal partitioning strategy.
+* `type` is set to `index_parallel`, meaning that parallel indexing will be used.
+* `segmentGranularity` is set to `DAY`, meaning that each compacted segment will be a day of data.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"\
+--header 'Content-Type: application/json' \
+--data '{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}'
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+Content-Type: application/json
+Content-Length: 281
+
+{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
+
+
+### Remove datasource automatic compaction configuration
+
+Removes the automatic compaction configuration for a datasource. This updates the compaction status of the datasource to "Not enabled."
+
+#### URL
+
+<code class="deleteAPI">DELETE</code> <code>/druid/coordinator/v1/config/compaction/:dataSource</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully deleted automatic compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Datasource does not have automatic compaction or invalid datasource name*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request DELETE "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+DELETE /druid/coordinator/v1/config/compaction/wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
## Automatic compaction configuration
-`GET /druid/coordinator/v1/config/compaction`
+### Get all automatic compaction configuration
+
+Retrieves all automatic compaction configurations. Returns a `compactionConfigs` object containing the active automatic compaction configuration of all datasources.
+
+You can use this endpoint to retrieve `compactionTaskSlotRatio` and `maxCompactionTaskSlots` values for managing resource allocation of compaction tasks.
+
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved automatic compaction configurations*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
-Returns all automatic compaction configs.
+```json
+{
+ "compactionConfigs": [
+ {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ {
+ "dataSource": "wikipedia",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ }
+ ],
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+}
+```
+</details>
-`GET /druid/coordinator/v1/config/compaction/{dataSource}`
+### Get datasource automatic compaction configuration
-Returns an automatic compaction config of a dataSource.
+Retrieves the automatic compaction configuration of a datasource.
-`GET /druid/coordinator/v1/config/compaction/{dataSource}/history?interval={interval}&count={count}`
+#### URL
-Returns the history of the automatic compaction config for a dataSource. Optionally accepts `interval` and `count`
-query string parameters to filter by interval and limit the number of results respectively. If the dataSource does not
-exist or there is no compaction history for the dataSource, an empty list is returned.
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction/:dataSource</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved configuration for datasource*
+
+<!--404 NOT FOUND-->
+
+*Invalid datasource or datasource does not have automatic compaction enabled*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+The following example retrieves the automatic compaction configuration for datasource `wikipedia_hour`.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction/wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
+
+```json
+{
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+}
+```
+</details>
+
+### Get datasource automatic compaction configuration history
+
+Retrieves the history of the automatic compaction config for a datasource. Returns an empty list if the datasource does not exist or there is no compaction history for the datasource.
The response contains a list of objects with the following keys:
-* `globalConfig`: A json object containing automatic compaction config that applies to the entire cluster.
-* `compactionConfig`: A json object containing the automatic compaction config for the datasource.
-* `auditInfo`: A json object that contains information about the change made - like `author`, `comment` and `ip`.
+* `globalConfig`: A JSON object containing automatic compaction config that applies to the entire cluster.
+* `compactionConfig`: A JSON object containing the automatic compaction config for the datasource.
+* `auditInfo`: A JSON object that contains information about the change made - like `author`, `comment` and `ip`.
* `auditTime`: The date and time when the change was made.
-`POST /druid/coordinator/v1/config/compaction/taskslots?ratio={someRatio}&max={someMaxSlots}`
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction/:dataSource/history</code>
+
+#### Query parameters
+* `interval` (optional)
+ * Type: ISO-8601
+ * Limit the results within a specified interval. Use `/` as the delimiter for the interval string.
+* `count` (optional)
+ * Type: Int
+ * Limit the number of results.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved configuration history*
+
+<!--400 BAD REQUEST-->
+
+*Invalid `count` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour/history"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction/wikipedia_hour/history HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
+
+```json
+[
+ {
+ "globalConfig": {
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+ },
+ "compactionConfig": {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "P1D",
+ "tuningConfig": null,
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ "auditInfo": {
+ "author": "",
+ "comment": "",
+ "ip": "127.0.0.1"
+ },
+ "auditTime": "2023-07-31T18:15:19.302Z"
+ },
+ {
+ "globalConfig": {
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+ },
+ "compactionConfig": {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ "auditInfo": {
+ "author": "",
+ "comment": "",
+ "ip": "127.0.0.1"
+ },
+ "auditTime": "2023-07-31T18:16:16.362Z"
+ }
+]
+```
+</details>
+
+## Automatic compaction status
+
+### Get segments awaiting compaction
+
+Returns the total size of segments awaiting compaction for the given datasource. The specified datasource must have automatic compaction enabled.
+
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/compaction/progress?dataSource=:dataSource</code>
+
+#### Query parameter
+* `dataSource` (required)
+ * Type: String
+ * Name of the datasource for this status information
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved segment size awaiting compaction*
+
+<!--404 NOT FOUND-->
+
+*Unknown datasource name or datasource does not have automatic compaction enabled*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+The following example retrieves the remaining segments to be compacted for datasource `wikipedia_hour`.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/compaction/progress?dataSource=wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/compaction/progress?dataSource=wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
+
+```json
+{
+ "remainingSegmentSize": 7615837
+}
+```
+</details>
+
+
+### Get compaction status and statistics
+
+Retrieves the status and statistics from the automatic compaction run of all datasources which have automatic compaction enabled in the latest run. Returns a list of `latestStatus` objects. Each `latestStatus` represents the status for a datasource with automatic compaction enabled.
+
+The `latestStatus` object has the following keys:
+* `dataSource`: Name of the datasource for this status information.
+* `scheduleStatus`: Automatic compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active automatic compaction config submitted. Otherwise, returns `NOT_ENABLED`.
+* `bytesAwaitingCompaction`: Total bytes of this datasource waiting to be compacted by the automatic compaction (only consider intervals/segments that are eligible for automatic compaction).
+* `bytesCompacted`: Total bytes of this datasource that are already compacted with the spec set in the automatic compaction config.
+* `bytesSkipped`: Total bytes of this datasource that are skipped (not eligible for automatic compaction) by the automatic compaction.
+* `segmentCountAwaitingCompaction`: Total number of segments of this datasource waiting to be compacted by the automatic compaction (only consider intervals/segments that are eligible for automatic compaction).
+* `segmentCountCompacted`: Total number of segments of this datasource that are already compacted with the spec set in the automatic compaction config.
+* `segmentCountSkipped`: Total number of segments of this datasource that are skipped (not eligible for automatic compaction) by the automatic compaction.
+* `intervalCountAwaitingCompaction`: Total number of intervals of this datasource waiting to be compacted by the automatic compaction (only consider intervals/segments that are eligible for automatic compaction).
+* `intervalCountCompacted`: Total number of intervals of this datasource that are already compacted with the spec set in the automatic compaction config.
Review Comment:
```suggestion
* `intervalCountCompacted`: Total number of intervals of this datasource that are already compacted with the spec set in the automatic compaction configuration.
```
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,762 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. Automatic compaction can be configured in the Druid web console or API.
-## Automatic compaction status
+In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments.
-`GET /druid/coordinator/v1/compaction/progress?dataSource={dataSource}`
+## Manage automatic compaction
-Returns the total size of segments awaiting compaction for the given dataSource. The specified dataSource must have [automatic compaction](../data-management/automatic-compaction.md) enabled.
+### Update capacity for compaction tasks
-`GET /druid/coordinator/v1/compaction/status`
+Update the capacity for compaction tasks. Maximum number of compaction tasks is limited by `ratio` and `max`.
+`ratio` is ratio of the total task slots to the compaction task slots and the `max` is the number of task slots for compaction tasks.
-Returns the status and statistics from the auto-compaction run of all dataSources which have auto-compaction enabled in the latest run. The response payload includes a list of `latestStatus` objects. Each `latestStatus` represents the status for a dataSource (which has/had auto-compaction enabled).
+The minimum number of compaction tasks is 1 and the maximum is 2147483652.
-The `latestStatus` object has the following keys:
-* `dataSource`: name of the datasource for this status information
-* `scheduleStatus`: auto-compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active auto-compaction config submitted. Otherwise, returns `NOT_ENABLED`.
-* `bytesAwaitingCompaction`: total bytes of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `bytesCompacted`: total bytes of this datasource that are already compacted with the spec set in the auto-compaction config
-* `bytesSkipped`: total bytes of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `segmentCountAwaitingCompaction`: total number of segments of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `segmentCountCompacted`: total number of segments of this datasource that are already compacted with the spec set in the auto-compaction config
-* `segmentCountSkipped`: total number of segments of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `intervalCountAwaitingCompaction`: total number of intervals of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `intervalCountCompacted`: total number of intervals of this datasource that are already compacted with the spec set in the auto-compaction config
-* `intervalCountSkipped`: total number of intervals of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-
-`GET /druid/coordinator/v1/compaction/status?dataSource={dataSource}`
-
-Similar to the API `/druid/coordinator/v1/compaction/status` above but filters response to only return information for the dataSource given.
-The dataSource must have auto-compaction enabled.
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction/taskslots</code>
+
+#### Query parameters
+* `ratio` (optional)
+ * Type: Float
+ * Default: 0.1
+ * Limit the ratio of the total task slots to compaction task slots.
+* `max` (optional)
+ * Type: Int
+ * Default: 2147483647
+ * Limit the maximum number of task slots for compaction tasks.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully updated compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Invalid `max` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000"
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000 HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
+
+### Create or update datasource automatic compaction configuration
+
+Creates or updates the automatic compaction config for a datasource. The automatic compaction can be submitted as a JSON object in the request body.
+
+Automatic compaction configuration require only the `dataSource` property. All others will be filled with default values if not specified. See [Automatic compaction dynamic configuration](../configuration/index.md#automatic-compaction-dynamic-configuration) for configuration details.
+
+Note that this endpoint will return an HTTP `200 OK` even if the datasource name does not exist.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully submitted auto compaction configuration*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+#### Sample request
+
+The following example creates an automatic compaction configuration for datasource `wikipedia_hour`. This automatic compaction configuration performs compaction on `wikipedia_hour`, resulting in compacted segments that represent a day interval of data.
+
+* `wikipedia_hour` is a datasource with `HOUR` segment granularity.
+* `skipOffsetFromLatest` is set to `PT0S`, meaning that no data is skipped.
+* `partitionsSpec` is set to the default `dynamic`, allowing Druid to dynamically determine the optimal partitioning strategy.
+* `type` is set to `index_parallel`, meaning that parallel indexing will be used.
+* `segmentGranularity` is set to `DAY`, meaning that each compacted segment will be a day of data.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"\
+--header 'Content-Type: application/json' \
+--data '{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}'
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+Content-Type: application/json
+Content-Length: 281
+
+{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
+
+
+### Remove datasource automatic compaction configuration
+
+Removes the automatic compaction configuration for a datasource. This updates the compaction status of the datasource to "Not enabled."
+
+#### URL
+
+<code class="deleteAPI">DELETE</code> <code>/druid/coordinator/v1/config/compaction/:dataSource</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully deleted automatic compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Datasource does not have automatic compaction or invalid datasource name*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request DELETE "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+DELETE /druid/coordinator/v1/config/compaction/wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
## Automatic compaction configuration
-`GET /druid/coordinator/v1/config/compaction`
+### Get all automatic compaction configuration
Review Comment:
```suggestion
### Get all automatic compaction configurations
```
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,762 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. Automatic compaction can be configured in the Druid web console or API.
-## Automatic compaction status
+In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments.
-`GET /druid/coordinator/v1/compaction/progress?dataSource={dataSource}`
+## Manage automatic compaction
-Returns the total size of segments awaiting compaction for the given dataSource. The specified dataSource must have [automatic compaction](../data-management/automatic-compaction.md) enabled.
+### Update capacity for compaction tasks
-`GET /druid/coordinator/v1/compaction/status`
+Update the capacity for compaction tasks. Maximum number of compaction tasks is limited by `ratio` and `max`.
+`ratio` is ratio of the total task slots to the compaction task slots and the `max` is the number of task slots for compaction tasks.
-Returns the status and statistics from the auto-compaction run of all dataSources which have auto-compaction enabled in the latest run. The response payload includes a list of `latestStatus` objects. Each `latestStatus` represents the status for a dataSource (which has/had auto-compaction enabled).
+The minimum number of compaction tasks is 1 and the maximum is 2147483652.
-The `latestStatus` object has the following keys:
-* `dataSource`: name of the datasource for this status information
-* `scheduleStatus`: auto-compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active auto-compaction config submitted. Otherwise, returns `NOT_ENABLED`.
-* `bytesAwaitingCompaction`: total bytes of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `bytesCompacted`: total bytes of this datasource that are already compacted with the spec set in the auto-compaction config
-* `bytesSkipped`: total bytes of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `segmentCountAwaitingCompaction`: total number of segments of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `segmentCountCompacted`: total number of segments of this datasource that are already compacted with the spec set in the auto-compaction config
-* `segmentCountSkipped`: total number of segments of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `intervalCountAwaitingCompaction`: total number of intervals of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `intervalCountCompacted`: total number of intervals of this datasource that are already compacted with the spec set in the auto-compaction config
-* `intervalCountSkipped`: total number of intervals of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-
-`GET /druid/coordinator/v1/compaction/status?dataSource={dataSource}`
-
-Similar to the API `/druid/coordinator/v1/compaction/status` above but filters response to only return information for the dataSource given.
-The dataSource must have auto-compaction enabled.
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction/taskslots</code>
+
+#### Query parameters
+* `ratio` (optional)
+ * Type: Float
+ * Default: 0.1
+ * Limit the ratio of the total task slots to compaction task slots.
+* `max` (optional)
+ * Type: Int
+ * Default: 2147483647
+ * Limit the maximum number of task slots for compaction tasks.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully updated compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Invalid `max` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000"
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000 HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
+
+### Create or update datasource automatic compaction configuration
+
+Creates or updates the automatic compaction config for a datasource. The automatic compaction can be submitted as a JSON object in the request body.
+
+Automatic compaction configuration require only the `dataSource` property. All others will be filled with default values if not specified. See [Automatic compaction dynamic configuration](../configuration/index.md#automatic-compaction-dynamic-configuration) for configuration details.
+
+Note that this endpoint will return an HTTP `200 OK` even if the datasource name does not exist.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully submitted auto compaction configuration*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+#### Sample request
+
+The following example creates an automatic compaction configuration for datasource `wikipedia_hour`. This automatic compaction configuration performs compaction on `wikipedia_hour`, resulting in compacted segments that represent a day interval of data.
+
+* `wikipedia_hour` is a datasource with `HOUR` segment granularity.
+* `skipOffsetFromLatest` is set to `PT0S`, meaning that no data is skipped.
+* `partitionsSpec` is set to the default `dynamic`, allowing Druid to dynamically determine the optimal partitioning strategy.
+* `type` is set to `index_parallel`, meaning that parallel indexing will be used.
+* `segmentGranularity` is set to `DAY`, meaning that each compacted segment will be a day of data.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"\
+--header 'Content-Type: application/json' \
+--data '{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}'
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+Content-Type: application/json
+Content-Length: 281
+
+{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
+
+
+### Remove datasource automatic compaction configuration
+
+Removes the automatic compaction configuration for a datasource. This updates the compaction status of the datasource to "Not enabled."
+
+#### URL
+
+<code class="deleteAPI">DELETE</code> <code>/druid/coordinator/v1/config/compaction/:dataSource</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully deleted automatic compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Datasource does not have automatic compaction or invalid datasource name*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request DELETE "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+DELETE /druid/coordinator/v1/config/compaction/wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
## Automatic compaction configuration
-`GET /druid/coordinator/v1/config/compaction`
+### Get all automatic compaction configuration
+
+Retrieves all automatic compaction configurations. Returns a `compactionConfigs` object containing the active automatic compaction configuration of all datasources.
+
+You can use this endpoint to retrieve `compactionTaskSlotRatio` and `maxCompactionTaskSlots` values for managing resource allocation of compaction tasks.
+
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved automatic compaction configurations*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
-Returns all automatic compaction configs.
+```json
+{
+ "compactionConfigs": [
+ {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ {
+ "dataSource": "wikipedia",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ }
+ ],
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+}
+```
+</details>
-`GET /druid/coordinator/v1/config/compaction/{dataSource}`
+### Get datasource automatic compaction configuration
-Returns an automatic compaction config of a dataSource.
+Retrieves the automatic compaction configuration of a datasource.
-`GET /druid/coordinator/v1/config/compaction/{dataSource}/history?interval={interval}&count={count}`
+#### URL
-Returns the history of the automatic compaction config for a dataSource. Optionally accepts `interval` and `count`
-query string parameters to filter by interval and limit the number of results respectively. If the dataSource does not
-exist or there is no compaction history for the dataSource, an empty list is returned.
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction/:dataSource</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved configuration for datasource*
+
+<!--404 NOT FOUND-->
+
+*Invalid datasource or datasource does not have automatic compaction enabled*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+The following example retrieves the automatic compaction configuration for datasource `wikipedia_hour`.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction/wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
+
+```json
+{
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+}
+```
+</details>
+
+### Get datasource automatic compaction configuration history
+
+Retrieves the history of the automatic compaction config for a datasource. Returns an empty list if the datasource does not exist or there is no compaction history for the datasource.
The response contains a list of objects with the following keys:
-* `globalConfig`: A json object containing automatic compaction config that applies to the entire cluster.
-* `compactionConfig`: A json object containing the automatic compaction config for the datasource.
-* `auditInfo`: A json object that contains information about the change made - like `author`, `comment` and `ip`.
+* `globalConfig`: A JSON object containing automatic compaction config that applies to the entire cluster.
+* `compactionConfig`: A JSON object containing the automatic compaction config for the datasource.
+* `auditInfo`: A JSON object that contains information about the change made - like `author`, `comment` and `ip`.
* `auditTime`: The date and time when the change was made.
-`POST /druid/coordinator/v1/config/compaction/taskslots?ratio={someRatio}&max={someMaxSlots}`
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction/:dataSource/history</code>
+
+#### Query parameters
+* `interval` (optional)
+ * Type: ISO-8601
+ * Limit the results within a specified interval. Use `/` as the delimiter for the interval string.
+* `count` (optional)
+ * Type: Int
+ * Limit the number of results.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved configuration history*
+
+<!--400 BAD REQUEST-->
+
+*Invalid `count` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour/history"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction/wikipedia_hour/history HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
+
+```json
+[
+ {
+ "globalConfig": {
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+ },
+ "compactionConfig": {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "P1D",
+ "tuningConfig": null,
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ "auditInfo": {
+ "author": "",
+ "comment": "",
+ "ip": "127.0.0.1"
+ },
+ "auditTime": "2023-07-31T18:15:19.302Z"
+ },
+ {
+ "globalConfig": {
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+ },
+ "compactionConfig": {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ "auditInfo": {
+ "author": "",
+ "comment": "",
+ "ip": "127.0.0.1"
+ },
+ "auditTime": "2023-07-31T18:16:16.362Z"
+ }
+]
+```
+</details>
+
+## Automatic compaction status
+
+### Get segments awaiting compaction
+
+Returns the total size of segments awaiting compaction for the given datasource. The specified datasource must have automatic compaction enabled.
+
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/compaction/progress?dataSource=:dataSource</code>
+
+#### Query parameter
+* `dataSource` (required)
+ * Type: String
+ * Name of the datasource for this status information
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved segment size awaiting compaction*
+
+<!--404 NOT FOUND-->
+
+*Unknown datasource name or datasource does not have automatic compaction enabled*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+The following example retrieves the remaining segments to be compacted for datasource `wikipedia_hour`.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/compaction/progress?dataSource=wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/compaction/progress?dataSource=wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
+
+```json
+{
+ "remainingSegmentSize": 7615837
+}
+```
+</details>
+
+
+### Get compaction status and statistics
+
+Retrieves the status and statistics from the automatic compaction run of all datasources which have automatic compaction enabled in the latest run. Returns a list of `latestStatus` objects. Each `latestStatus` represents the status for a datasource with automatic compaction enabled.
+
+The `latestStatus` object has the following keys:
+* `dataSource`: Name of the datasource for this status information.
+* `scheduleStatus`: Automatic compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active automatic compaction config submitted. Otherwise, returns `NOT_ENABLED`.
+* `bytesAwaitingCompaction`: Total bytes of this datasource waiting to be compacted by the automatic compaction (only consider intervals/segments that are eligible for automatic compaction).
+* `bytesCompacted`: Total bytes of this datasource that are already compacted with the spec set in the automatic compaction config.
+* `bytesSkipped`: Total bytes of this datasource that are skipped (not eligible for automatic compaction) by the automatic compaction.
+* `segmentCountAwaitingCompaction`: Total number of segments of this datasource waiting to be compacted by the automatic compaction (only consider intervals/segments that are eligible for automatic compaction).
+* `segmentCountCompacted`: Total number of segments of this datasource that are already compacted with the spec set in the automatic compaction config.
+* `segmentCountSkipped`: Total number of segments of this datasource that are skipped (not eligible for automatic compaction) by the automatic compaction.
+* `intervalCountAwaitingCompaction`: Total number of intervals of this datasource waiting to be compacted by the automatic compaction (only consider intervals/segments that are eligible for automatic compaction).
+* `intervalCountCompacted`: Total number of intervals of this datasource that are already compacted with the spec set in the automatic compaction config.
+* `intervalCountSkipped`: Total number of intervals of this datasource that are skipped (not eligible for automatic compaction) by the automatic compaction.
+
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/compaction/status</code>
+
+#### Query parameters
+* `dataSource` (optional)
+ * Type: String
+ * Filter the result by name of specific datasource.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved `latestStatus` object*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "hhttp://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/compaction/status"
Review Comment:
```suggestion
curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/compaction/status"
```
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,762 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. Automatic compaction can be configured in the Druid web console or API.
-## Automatic compaction status
+In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments.
-`GET /druid/coordinator/v1/compaction/progress?dataSource={dataSource}`
+## Manage automatic compaction
-Returns the total size of segments awaiting compaction for the given dataSource. The specified dataSource must have [automatic compaction](../data-management/automatic-compaction.md) enabled.
+### Update capacity for compaction tasks
-`GET /druid/coordinator/v1/compaction/status`
+Update the capacity for compaction tasks. Maximum number of compaction tasks is limited by `ratio` and `max`.
+`ratio` is ratio of the total task slots to the compaction task slots and the `max` is the number of task slots for compaction tasks.
-Returns the status and statistics from the auto-compaction run of all dataSources which have auto-compaction enabled in the latest run. The response payload includes a list of `latestStatus` objects. Each `latestStatus` represents the status for a dataSource (which has/had auto-compaction enabled).
+The minimum number of compaction tasks is 1 and the maximum is 2147483652.
-The `latestStatus` object has the following keys:
-* `dataSource`: name of the datasource for this status information
-* `scheduleStatus`: auto-compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active auto-compaction config submitted. Otherwise, returns `NOT_ENABLED`.
-* `bytesAwaitingCompaction`: total bytes of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `bytesCompacted`: total bytes of this datasource that are already compacted with the spec set in the auto-compaction config
-* `bytesSkipped`: total bytes of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `segmentCountAwaitingCompaction`: total number of segments of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `segmentCountCompacted`: total number of segments of this datasource that are already compacted with the spec set in the auto-compaction config
-* `segmentCountSkipped`: total number of segments of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `intervalCountAwaitingCompaction`: total number of intervals of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `intervalCountCompacted`: total number of intervals of this datasource that are already compacted with the spec set in the auto-compaction config
-* `intervalCountSkipped`: total number of intervals of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-
-`GET /druid/coordinator/v1/compaction/status?dataSource={dataSource}`
-
-Similar to the API `/druid/coordinator/v1/compaction/status` above but filters response to only return information for the dataSource given.
-The dataSource must have auto-compaction enabled.
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction/taskslots</code>
+
+#### Query parameters
+* `ratio` (optional)
+ * Type: Float
+ * Default: 0.1
+ * Limit the ratio of the total task slots to compaction task slots.
+* `max` (optional)
+ * Type: Int
+ * Default: 2147483647
+ * Limit the maximum number of task slots for compaction tasks.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully updated compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Invalid `max` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000"
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000 HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
+
+### Create or update datasource automatic compaction configuration
+
+Creates or updates the automatic compaction config for a datasource. The automatic compaction can be submitted as a JSON object in the request body.
+
+Automatic compaction configuration require only the `dataSource` property. All others will be filled with default values if not specified. See [Automatic compaction dynamic configuration](../configuration/index.md#automatic-compaction-dynamic-configuration) for configuration details.
+
+Note that this endpoint will return an HTTP `200 OK` even if the datasource name does not exist.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully submitted auto compaction configuration*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+#### Sample request
+
+The following example creates an automatic compaction configuration for datasource `wikipedia_hour`. This automatic compaction configuration performs compaction on `wikipedia_hour`, resulting in compacted segments that represent a day interval of data.
+
+* `wikipedia_hour` is a datasource with `HOUR` segment granularity.
+* `skipOffsetFromLatest` is set to `PT0S`, meaning that no data is skipped.
+* `partitionsSpec` is set to the default `dynamic`, allowing Druid to dynamically determine the optimal partitioning strategy.
+* `type` is set to `index_parallel`, meaning that parallel indexing will be used.
+* `segmentGranularity` is set to `DAY`, meaning that each compacted segment will be a day of data.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"\
+--header 'Content-Type: application/json' \
+--data '{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}'
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+Content-Type: application/json
+Content-Length: 281
+
+{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
+
+
+### Remove datasource automatic compaction configuration
+
+Removes the automatic compaction configuration for a datasource. This updates the compaction status of the datasource to "Not enabled."
+
+#### URL
+
+<code class="deleteAPI">DELETE</code> <code>/druid/coordinator/v1/config/compaction/:dataSource</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully deleted automatic compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Datasource does not have automatic compaction or invalid datasource name*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request DELETE "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+DELETE /druid/coordinator/v1/config/compaction/wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
## Automatic compaction configuration
-`GET /druid/coordinator/v1/config/compaction`
+### Get all automatic compaction configuration
+
+Retrieves all automatic compaction configurations. Returns a `compactionConfigs` object containing the active automatic compaction configuration of all datasources.
+
+You can use this endpoint to retrieve `compactionTaskSlotRatio` and `maxCompactionTaskSlots` values for managing resource allocation of compaction tasks.
+
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved automatic compaction configurations*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
-Returns all automatic compaction configs.
+```json
+{
+ "compactionConfigs": [
+ {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ {
+ "dataSource": "wikipedia",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ }
+ ],
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+}
+```
+</details>
-`GET /druid/coordinator/v1/config/compaction/{dataSource}`
+### Get datasource automatic compaction configuration
-Returns an automatic compaction config of a dataSource.
+Retrieves the automatic compaction configuration of a datasource.
-`GET /druid/coordinator/v1/config/compaction/{dataSource}/history?interval={interval}&count={count}`
+#### URL
-Returns the history of the automatic compaction config for a dataSource. Optionally accepts `interval` and `count`
-query string parameters to filter by interval and limit the number of results respectively. If the dataSource does not
-exist or there is no compaction history for the dataSource, an empty list is returned.
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction/:dataSource</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved configuration for datasource*
+
+<!--404 NOT FOUND-->
+
+*Invalid datasource or datasource does not have automatic compaction enabled*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+The following example retrieves the automatic compaction configuration for datasource `wikipedia_hour`.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction/wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
+
+```json
+{
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+}
+```
+</details>
+
+### Get datasource automatic compaction configuration history
+
+Retrieves the history of the automatic compaction config for a datasource. Returns an empty list if the datasource does not exist or there is no compaction history for the datasource.
The response contains a list of objects with the following keys:
-* `globalConfig`: A json object containing automatic compaction config that applies to the entire cluster.
-* `compactionConfig`: A json object containing the automatic compaction config for the datasource.
-* `auditInfo`: A json object that contains information about the change made - like `author`, `comment` and `ip`.
+* `globalConfig`: A JSON object containing automatic compaction config that applies to the entire cluster.
Review Comment:
```suggestion
* `globalConfig`: A JSON object containing automatic compaction configuration that applies to the entire cluster.
```
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,762 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. Automatic compaction can be configured in the Druid web console or API.
-## Automatic compaction status
+In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments.
-`GET /druid/coordinator/v1/compaction/progress?dataSource={dataSource}`
+## Manage automatic compaction
-Returns the total size of segments awaiting compaction for the given dataSource. The specified dataSource must have [automatic compaction](../data-management/automatic-compaction.md) enabled.
+### Update capacity for compaction tasks
-`GET /druid/coordinator/v1/compaction/status`
+Update the capacity for compaction tasks. Maximum number of compaction tasks is limited by `ratio` and `max`.
+`ratio` is ratio of the total task slots to the compaction task slots and the `max` is the number of task slots for compaction tasks.
-Returns the status and statistics from the auto-compaction run of all dataSources which have auto-compaction enabled in the latest run. The response payload includes a list of `latestStatus` objects. Each `latestStatus` represents the status for a dataSource (which has/had auto-compaction enabled).
+The minimum number of compaction tasks is 1 and the maximum is 2147483652.
-The `latestStatus` object has the following keys:
-* `dataSource`: name of the datasource for this status information
-* `scheduleStatus`: auto-compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active auto-compaction config submitted. Otherwise, returns `NOT_ENABLED`.
-* `bytesAwaitingCompaction`: total bytes of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `bytesCompacted`: total bytes of this datasource that are already compacted with the spec set in the auto-compaction config
-* `bytesSkipped`: total bytes of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `segmentCountAwaitingCompaction`: total number of segments of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `segmentCountCompacted`: total number of segments of this datasource that are already compacted with the spec set in the auto-compaction config
-* `segmentCountSkipped`: total number of segments of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `intervalCountAwaitingCompaction`: total number of intervals of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `intervalCountCompacted`: total number of intervals of this datasource that are already compacted with the spec set in the auto-compaction config
-* `intervalCountSkipped`: total number of intervals of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-
-`GET /druid/coordinator/v1/compaction/status?dataSource={dataSource}`
-
-Similar to the API `/druid/coordinator/v1/compaction/status` above but filters response to only return information for the dataSource given.
-The dataSource must have auto-compaction enabled.
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction/taskslots</code>
+
+#### Query parameters
+* `ratio` (optional)
+ * Type: Float
+ * Default: 0.1
+ * Limit the ratio of the total task slots to compaction task slots.
+* `max` (optional)
+ * Type: Int
+ * Default: 2147483647
+ * Limit the maximum number of task slots for compaction tasks.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully updated compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Invalid `max` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000"
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000 HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
+
+### Create or update datasource automatic compaction configuration
+
+Creates or updates the automatic compaction config for a datasource. The automatic compaction can be submitted as a JSON object in the request body.
+
+Automatic compaction configuration require only the `dataSource` property. All others will be filled with default values if not specified. See [Automatic compaction dynamic configuration](../configuration/index.md#automatic-compaction-dynamic-configuration) for configuration details.
+
+Note that this endpoint will return an HTTP `200 OK` even if the datasource name does not exist.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully submitted auto compaction configuration*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+#### Sample request
+
+The following example creates an automatic compaction configuration for datasource `wikipedia_hour`. This automatic compaction configuration performs compaction on `wikipedia_hour`, resulting in compacted segments that represent a day interval of data.
+
+* `wikipedia_hour` is a datasource with `HOUR` segment granularity.
+* `skipOffsetFromLatest` is set to `PT0S`, meaning that no data is skipped.
+* `partitionsSpec` is set to the default `dynamic`, allowing Druid to dynamically determine the optimal partitioning strategy.
+* `type` is set to `index_parallel`, meaning that parallel indexing will be used.
+* `segmentGranularity` is set to `DAY`, meaning that each compacted segment will be a day of data.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"\
+--header 'Content-Type: application/json' \
+--data '{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}'
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+Content-Type: application/json
+Content-Length: 281
+
+{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
+
+
+### Remove datasource automatic compaction configuration
+
+Removes the automatic compaction configuration for a datasource. This updates the compaction status of the datasource to "Not enabled."
+
+#### URL
+
+<code class="deleteAPI">DELETE</code> <code>/druid/coordinator/v1/config/compaction/:dataSource</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully deleted automatic compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Datasource does not have automatic compaction or invalid datasource name*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request DELETE "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+DELETE /druid/coordinator/v1/config/compaction/wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
## Automatic compaction configuration
-`GET /druid/coordinator/v1/config/compaction`
+### Get all automatic compaction configuration
+
+Retrieves all automatic compaction configurations. Returns a `compactionConfigs` object containing the active automatic compaction configuration of all datasources.
+
+You can use this endpoint to retrieve `compactionTaskSlotRatio` and `maxCompactionTaskSlots` values for managing resource allocation of compaction tasks.
+
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved automatic compaction configurations*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
-Returns all automatic compaction configs.
+```json
+{
+ "compactionConfigs": [
+ {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ {
+ "dataSource": "wikipedia",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ }
+ ],
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+}
+```
+</details>
-`GET /druid/coordinator/v1/config/compaction/{dataSource}`
+### Get datasource automatic compaction configuration
-Returns an automatic compaction config of a dataSource.
+Retrieves the automatic compaction configuration of a datasource.
-`GET /druid/coordinator/v1/config/compaction/{dataSource}/history?interval={interval}&count={count}`
+#### URL
-Returns the history of the automatic compaction config for a dataSource. Optionally accepts `interval` and `count`
-query string parameters to filter by interval and limit the number of results respectively. If the dataSource does not
-exist or there is no compaction history for the dataSource, an empty list is returned.
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction/:dataSource</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved configuration for datasource*
+
+<!--404 NOT FOUND-->
+
+*Invalid datasource or datasource does not have automatic compaction enabled*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+The following example retrieves the automatic compaction configuration for datasource `wikipedia_hour`.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction/wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
+
+```json
+{
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+}
+```
+</details>
+
+### Get datasource automatic compaction configuration history
+
+Retrieves the history of the automatic compaction config for a datasource. Returns an empty list if the datasource does not exist or there is no compaction history for the datasource.
The response contains a list of objects with the following keys:
-* `globalConfig`: A json object containing automatic compaction config that applies to the entire cluster.
-* `compactionConfig`: A json object containing the automatic compaction config for the datasource.
-* `auditInfo`: A json object that contains information about the change made - like `author`, `comment` and `ip`.
+* `globalConfig`: A JSON object containing automatic compaction config that applies to the entire cluster.
+* `compactionConfig`: A JSON object containing the automatic compaction config for the datasource.
+* `auditInfo`: A JSON object that contains information about the change made - like `author`, `comment` and `ip`.
Review Comment:
```suggestion
* `auditInfo`: A JSON object that contains information about the change, such as `author`, `comment`, or `ip`.
```
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,762 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. Automatic compaction can be configured in the Druid web console or API.
-## Automatic compaction status
+In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments.
-`GET /druid/coordinator/v1/compaction/progress?dataSource={dataSource}`
+## Manage automatic compaction
-Returns the total size of segments awaiting compaction for the given dataSource. The specified dataSource must have [automatic compaction](../data-management/automatic-compaction.md) enabled.
+### Update capacity for compaction tasks
-`GET /druid/coordinator/v1/compaction/status`
+Update the capacity for compaction tasks. Maximum number of compaction tasks is limited by `ratio` and `max`.
+`ratio` is ratio of the total task slots to the compaction task slots and the `max` is the number of task slots for compaction tasks.
-Returns the status and statistics from the auto-compaction run of all dataSources which have auto-compaction enabled in the latest run. The response payload includes a list of `latestStatus` objects. Each `latestStatus` represents the status for a dataSource (which has/had auto-compaction enabled).
+The minimum number of compaction tasks is 1 and the maximum is 2147483652.
-The `latestStatus` object has the following keys:
-* `dataSource`: name of the datasource for this status information
-* `scheduleStatus`: auto-compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active auto-compaction config submitted. Otherwise, returns `NOT_ENABLED`.
-* `bytesAwaitingCompaction`: total bytes of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `bytesCompacted`: total bytes of this datasource that are already compacted with the spec set in the auto-compaction config
-* `bytesSkipped`: total bytes of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `segmentCountAwaitingCompaction`: total number of segments of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `segmentCountCompacted`: total number of segments of this datasource that are already compacted with the spec set in the auto-compaction config
-* `segmentCountSkipped`: total number of segments of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `intervalCountAwaitingCompaction`: total number of intervals of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `intervalCountCompacted`: total number of intervals of this datasource that are already compacted with the spec set in the auto-compaction config
-* `intervalCountSkipped`: total number of intervals of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-
-`GET /druid/coordinator/v1/compaction/status?dataSource={dataSource}`
-
-Similar to the API `/druid/coordinator/v1/compaction/status` above but filters response to only return information for the dataSource given.
-The dataSource must have auto-compaction enabled.
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction/taskslots</code>
+
+#### Query parameters
+* `ratio` (optional)
+ * Type: Float
+ * Default: 0.1
+ * Limit the ratio of the total task slots to compaction task slots.
+* `max` (optional)
+ * Type: Int
+ * Default: 2147483647
+ * Limit the maximum number of task slots for compaction tasks.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully updated compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Invalid `max` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000"
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000 HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
+
+### Create or update datasource automatic compaction configuration
+
+Creates or updates the automatic compaction config for a datasource. The automatic compaction can be submitted as a JSON object in the request body.
+
+Automatic compaction configuration require only the `dataSource` property. All others will be filled with default values if not specified. See [Automatic compaction dynamic configuration](../configuration/index.md#automatic-compaction-dynamic-configuration) for configuration details.
+
+Note that this endpoint will return an HTTP `200 OK` even if the datasource name does not exist.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully submitted auto compaction configuration*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+#### Sample request
+
+The following example creates an automatic compaction configuration for datasource `wikipedia_hour`. This automatic compaction configuration performs compaction on `wikipedia_hour`, resulting in compacted segments that represent a day interval of data.
+
+* `wikipedia_hour` is a datasource with `HOUR` segment granularity.
+* `skipOffsetFromLatest` is set to `PT0S`, meaning that no data is skipped.
+* `partitionsSpec` is set to the default `dynamic`, allowing Druid to dynamically determine the optimal partitioning strategy.
+* `type` is set to `index_parallel`, meaning that parallel indexing will be used.
+* `segmentGranularity` is set to `DAY`, meaning that each compacted segment will be a day of data.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"\
+--header 'Content-Type: application/json' \
+--data '{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}'
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+Content-Type: application/json
+Content-Length: 281
+
+{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
+
+
+### Remove datasource automatic compaction configuration
+
+Removes the automatic compaction configuration for a datasource. This updates the compaction status of the datasource to "Not enabled."
+
+#### URL
+
+<code class="deleteAPI">DELETE</code> <code>/druid/coordinator/v1/config/compaction/:dataSource</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully deleted automatic compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Datasource does not have automatic compaction or invalid datasource name*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request DELETE "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+DELETE /druid/coordinator/v1/config/compaction/wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
## Automatic compaction configuration
-`GET /druid/coordinator/v1/config/compaction`
+### Get all automatic compaction configuration
+
+Retrieves all automatic compaction configurations. Returns a `compactionConfigs` object containing the active automatic compaction configuration of all datasources.
+
+You can use this endpoint to retrieve `compactionTaskSlotRatio` and `maxCompactionTaskSlots` values for managing resource allocation of compaction tasks.
+
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved automatic compaction configurations*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
-Returns all automatic compaction configs.
+```json
+{
+ "compactionConfigs": [
+ {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ {
+ "dataSource": "wikipedia",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ }
+ ],
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+}
+```
+</details>
-`GET /druid/coordinator/v1/config/compaction/{dataSource}`
+### Get datasource automatic compaction configuration
-Returns an automatic compaction config of a dataSource.
+Retrieves the automatic compaction configuration of a datasource.
-`GET /druid/coordinator/v1/config/compaction/{dataSource}/history?interval={interval}&count={count}`
+#### URL
-Returns the history of the automatic compaction config for a dataSource. Optionally accepts `interval` and `count`
-query string parameters to filter by interval and limit the number of results respectively. If the dataSource does not
-exist or there is no compaction history for the dataSource, an empty list is returned.
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction/:dataSource</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved configuration for datasource*
+
+<!--404 NOT FOUND-->
+
+*Invalid datasource or datasource does not have automatic compaction enabled*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+The following example retrieves the automatic compaction configuration for datasource `wikipedia_hour`.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction/wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
+
+```json
+{
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+}
+```
+</details>
+
+### Get datasource automatic compaction configuration history
+
+Retrieves the history of the automatic compaction config for a datasource. Returns an empty list if the datasource does not exist or there is no compaction history for the datasource.
The response contains a list of objects with the following keys:
-* `globalConfig`: A json object containing automatic compaction config that applies to the entire cluster.
-* `compactionConfig`: A json object containing the automatic compaction config for the datasource.
-* `auditInfo`: A json object that contains information about the change made - like `author`, `comment` and `ip`.
+* `globalConfig`: A JSON object containing automatic compaction config that applies to the entire cluster.
+* `compactionConfig`: A JSON object containing the automatic compaction config for the datasource.
+* `auditInfo`: A JSON object that contains information about the change made - like `author`, `comment` and `ip`.
* `auditTime`: The date and time when the change was made.
-`POST /druid/coordinator/v1/config/compaction/taskslots?ratio={someRatio}&max={someMaxSlots}`
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction/:dataSource/history</code>
+
+#### Query parameters
+* `interval` (optional)
+ * Type: ISO-8601
+ * Limit the results within a specified interval. Use `/` as the delimiter for the interval string.
+* `count` (optional)
+ * Type: Int
+ * Limit the number of results.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved configuration history*
+
+<!--400 BAD REQUEST-->
+
+*Invalid `count` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour/history"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction/wikipedia_hour/history HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
+
+```json
+[
+ {
+ "globalConfig": {
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+ },
+ "compactionConfig": {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "P1D",
+ "tuningConfig": null,
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ "auditInfo": {
+ "author": "",
+ "comment": "",
+ "ip": "127.0.0.1"
+ },
+ "auditTime": "2023-07-31T18:15:19.302Z"
+ },
+ {
+ "globalConfig": {
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+ },
+ "compactionConfig": {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ "auditInfo": {
+ "author": "",
+ "comment": "",
+ "ip": "127.0.0.1"
+ },
+ "auditTime": "2023-07-31T18:16:16.362Z"
+ }
+]
+```
+</details>
+
+## Automatic compaction status
+
+### Get segments awaiting compaction
+
+Returns the total size of segments awaiting compaction for the given datasource. The specified datasource must have automatic compaction enabled.
Review Comment:
```suggestion
Returns the total size of segments awaiting compaction for a given datasource. The specified datasource must have automatic compaction enabled.
```
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,762 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. Automatic compaction can be configured in the Druid web console or API.
-## Automatic compaction status
+In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments.
-`GET /druid/coordinator/v1/compaction/progress?dataSource={dataSource}`
+## Manage automatic compaction
-Returns the total size of segments awaiting compaction for the given dataSource. The specified dataSource must have [automatic compaction](../data-management/automatic-compaction.md) enabled.
+### Update capacity for compaction tasks
-`GET /druid/coordinator/v1/compaction/status`
+Update the capacity for compaction tasks. Maximum number of compaction tasks is limited by `ratio` and `max`.
+`ratio` is ratio of the total task slots to the compaction task slots and the `max` is the number of task slots for compaction tasks.
-Returns the status and statistics from the auto-compaction run of all dataSources which have auto-compaction enabled in the latest run. The response payload includes a list of `latestStatus` objects. Each `latestStatus` represents the status for a dataSource (which has/had auto-compaction enabled).
+The minimum number of compaction tasks is 1 and the maximum is 2147483652.
-The `latestStatus` object has the following keys:
-* `dataSource`: name of the datasource for this status information
-* `scheduleStatus`: auto-compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active auto-compaction config submitted. Otherwise, returns `NOT_ENABLED`.
-* `bytesAwaitingCompaction`: total bytes of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `bytesCompacted`: total bytes of this datasource that are already compacted with the spec set in the auto-compaction config
-* `bytesSkipped`: total bytes of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `segmentCountAwaitingCompaction`: total number of segments of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `segmentCountCompacted`: total number of segments of this datasource that are already compacted with the spec set in the auto-compaction config
-* `segmentCountSkipped`: total number of segments of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `intervalCountAwaitingCompaction`: total number of intervals of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `intervalCountCompacted`: total number of intervals of this datasource that are already compacted with the spec set in the auto-compaction config
-* `intervalCountSkipped`: total number of intervals of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-
-`GET /druid/coordinator/v1/compaction/status?dataSource={dataSource}`
-
-Similar to the API `/druid/coordinator/v1/compaction/status` above but filters response to only return information for the dataSource given.
-The dataSource must have auto-compaction enabled.
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction/taskslots</code>
+
+#### Query parameters
+* `ratio` (optional)
+ * Type: Float
+ * Default: 0.1
+ * Limit the ratio of the total task slots to compaction task slots.
+* `max` (optional)
+ * Type: Int
+ * Default: 2147483647
+ * Limit the maximum number of task slots for compaction tasks.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully updated compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Invalid `max` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000"
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000 HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
+
+### Create or update datasource automatic compaction configuration
+
+Creates or updates the automatic compaction config for a datasource. The automatic compaction can be submitted as a JSON object in the request body.
+
+Automatic compaction configuration require only the `dataSource` property. All others will be filled with default values if not specified. See [Automatic compaction dynamic configuration](../configuration/index.md#automatic-compaction-dynamic-configuration) for configuration details.
+
+Note that this endpoint will return an HTTP `200 OK` even if the datasource name does not exist.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully submitted auto compaction configuration*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+#### Sample request
+
+The following example creates an automatic compaction configuration for datasource `wikipedia_hour`. This automatic compaction configuration performs compaction on `wikipedia_hour`, resulting in compacted segments that represent a day interval of data.
+
+* `wikipedia_hour` is a datasource with `HOUR` segment granularity.
+* `skipOffsetFromLatest` is set to `PT0S`, meaning that no data is skipped.
+* `partitionsSpec` is set to the default `dynamic`, allowing Druid to dynamically determine the optimal partitioning strategy.
+* `type` is set to `index_parallel`, meaning that parallel indexing will be used.
+* `segmentGranularity` is set to `DAY`, meaning that each compacted segment will be a day of data.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"\
+--header 'Content-Type: application/json' \
+--data '{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}'
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+Content-Type: application/json
+Content-Length: 281
+
+{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
+
+
+### Remove datasource automatic compaction configuration
+
+Removes the automatic compaction configuration for a datasource. This updates the compaction status of the datasource to "Not enabled."
+
+#### URL
+
+<code class="deleteAPI">DELETE</code> <code>/druid/coordinator/v1/config/compaction/:dataSource</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully deleted automatic compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Datasource does not have automatic compaction or invalid datasource name*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request DELETE "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+DELETE /druid/coordinator/v1/config/compaction/wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
## Automatic compaction configuration
-`GET /druid/coordinator/v1/config/compaction`
+### Get all automatic compaction configuration
+
+Retrieves all automatic compaction configurations. Returns a `compactionConfigs` object containing the active automatic compaction configuration of all datasources.
+
+You can use this endpoint to retrieve `compactionTaskSlotRatio` and `maxCompactionTaskSlots` values for managing resource allocation of compaction tasks.
+
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved automatic compaction configurations*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
-Returns all automatic compaction configs.
+```json
+{
+ "compactionConfigs": [
+ {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ {
+ "dataSource": "wikipedia",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ }
+ ],
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+}
+```
+</details>
-`GET /druid/coordinator/v1/config/compaction/{dataSource}`
+### Get datasource automatic compaction configuration
-Returns an automatic compaction config of a dataSource.
+Retrieves the automatic compaction configuration of a datasource.
-`GET /druid/coordinator/v1/config/compaction/{dataSource}/history?interval={interval}&count={count}`
+#### URL
-Returns the history of the automatic compaction config for a dataSource. Optionally accepts `interval` and `count`
-query string parameters to filter by interval and limit the number of results respectively. If the dataSource does not
-exist or there is no compaction history for the dataSource, an empty list is returned.
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction/:dataSource</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved configuration for datasource*
+
+<!--404 NOT FOUND-->
+
+*Invalid datasource or datasource does not have automatic compaction enabled*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+The following example retrieves the automatic compaction configuration for datasource `wikipedia_hour`.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction/wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
+
+```json
+{
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+}
+```
+</details>
+
+### Get datasource automatic compaction configuration history
+
+Retrieves the history of the automatic compaction config for a datasource. Returns an empty list if the datasource does not exist or there is no compaction history for the datasource.
The response contains a list of objects with the following keys:
-* `globalConfig`: A json object containing automatic compaction config that applies to the entire cluster.
-* `compactionConfig`: A json object containing the automatic compaction config for the datasource.
-* `auditInfo`: A json object that contains information about the change made - like `author`, `comment` and `ip`.
+* `globalConfig`: A JSON object containing automatic compaction config that applies to the entire cluster.
+* `compactionConfig`: A JSON object containing the automatic compaction config for the datasource.
+* `auditInfo`: A JSON object that contains information about the change made - like `author`, `comment` and `ip`.
* `auditTime`: The date and time when the change was made.
-`POST /druid/coordinator/v1/config/compaction/taskslots?ratio={someRatio}&max={someMaxSlots}`
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction/:dataSource/history</code>
+
+#### Query parameters
+* `interval` (optional)
+ * Type: ISO-8601
+ * Limit the results within a specified interval. Use `/` as the delimiter for the interval string.
+* `count` (optional)
+ * Type: Int
+ * Limit the number of results.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved configuration history*
+
+<!--400 BAD REQUEST-->
+
+*Invalid `count` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour/history"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction/wikipedia_hour/history HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
+
+```json
+[
+ {
+ "globalConfig": {
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+ },
+ "compactionConfig": {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "P1D",
+ "tuningConfig": null,
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ "auditInfo": {
+ "author": "",
+ "comment": "",
+ "ip": "127.0.0.1"
+ },
+ "auditTime": "2023-07-31T18:15:19.302Z"
+ },
+ {
+ "globalConfig": {
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+ },
+ "compactionConfig": {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ "auditInfo": {
+ "author": "",
+ "comment": "",
+ "ip": "127.0.0.1"
+ },
+ "auditTime": "2023-07-31T18:16:16.362Z"
+ }
+]
+```
+</details>
+
+## Automatic compaction status
+
+### Get segments awaiting compaction
+
+Returns the total size of segments awaiting compaction for the given datasource. The specified datasource must have automatic compaction enabled.
+
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/compaction/progress?dataSource=:dataSource</code>
+
+#### Query parameter
+* `dataSource` (required)
+ * Type: String
+ * Name of the datasource for this status information
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved segment size awaiting compaction*
+
+<!--404 NOT FOUND-->
+
+*Unknown datasource name or datasource does not have automatic compaction enabled*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+The following example retrieves the remaining segments to be compacted for datasource `wikipedia_hour`.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/compaction/progress?dataSource=wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/compaction/progress?dataSource=wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
+
+```json
+{
+ "remainingSegmentSize": 7615837
+}
+```
+</details>
+
+
+### Get compaction status and statistics
+
+Retrieves the status and statistics from the automatic compaction run of all datasources which have automatic compaction enabled in the latest run. Returns a list of `latestStatus` objects. Each `latestStatus` represents the status for a datasource with automatic compaction enabled.
+
+The `latestStatus` object has the following keys:
+* `dataSource`: Name of the datasource for this status information.
+* `scheduleStatus`: Automatic compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active automatic compaction config submitted. Otherwise, returns `NOT_ENABLED`.
Review Comment:
```suggestion
* `scheduleStatus`: Automatic compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the datasource has an active automatic compaction configuration submitted. Otherwise, returns `NOT_ENABLED`.
```
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,762 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. Automatic compaction can be configured in the Druid web console or API.
-## Automatic compaction status
+In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments.
-`GET /druid/coordinator/v1/compaction/progress?dataSource={dataSource}`
+## Manage automatic compaction
-Returns the total size of segments awaiting compaction for the given dataSource. The specified dataSource must have [automatic compaction](../data-management/automatic-compaction.md) enabled.
+### Update capacity for compaction tasks
-`GET /druid/coordinator/v1/compaction/status`
+Update the capacity for compaction tasks. Maximum number of compaction tasks is limited by `ratio` and `max`.
+`ratio` is ratio of the total task slots to the compaction task slots and the `max` is the number of task slots for compaction tasks.
-Returns the status and statistics from the auto-compaction run of all dataSources which have auto-compaction enabled in the latest run. The response payload includes a list of `latestStatus` objects. Each `latestStatus` represents the status for a dataSource (which has/had auto-compaction enabled).
+The minimum number of compaction tasks is 1 and the maximum is 2147483652.
-The `latestStatus` object has the following keys:
-* `dataSource`: name of the datasource for this status information
-* `scheduleStatus`: auto-compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active auto-compaction config submitted. Otherwise, returns `NOT_ENABLED`.
-* `bytesAwaitingCompaction`: total bytes of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `bytesCompacted`: total bytes of this datasource that are already compacted with the spec set in the auto-compaction config
-* `bytesSkipped`: total bytes of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `segmentCountAwaitingCompaction`: total number of segments of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `segmentCountCompacted`: total number of segments of this datasource that are already compacted with the spec set in the auto-compaction config
-* `segmentCountSkipped`: total number of segments of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `intervalCountAwaitingCompaction`: total number of intervals of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `intervalCountCompacted`: total number of intervals of this datasource that are already compacted with the spec set in the auto-compaction config
-* `intervalCountSkipped`: total number of intervals of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-
-`GET /druid/coordinator/v1/compaction/status?dataSource={dataSource}`
-
-Similar to the API `/druid/coordinator/v1/compaction/status` above but filters response to only return information for the dataSource given.
-The dataSource must have auto-compaction enabled.
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction/taskslots</code>
+
+#### Query parameters
+* `ratio` (optional)
+ * Type: Float
+ * Default: 0.1
+ * Limit the ratio of the total task slots to compaction task slots.
+* `max` (optional)
+ * Type: Int
+ * Default: 2147483647
+ * Limit the maximum number of task slots for compaction tasks.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully updated compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Invalid `max` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000"
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000 HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
+
+### Create or update datasource automatic compaction configuration
+
+Creates or updates the automatic compaction config for a datasource. The automatic compaction can be submitted as a JSON object in the request body.
+
+Automatic compaction configuration require only the `dataSource` property. All others will be filled with default values if not specified. See [Automatic compaction dynamic configuration](../configuration/index.md#automatic-compaction-dynamic-configuration) for configuration details.
+
+Note that this endpoint will return an HTTP `200 OK` even if the datasource name does not exist.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully submitted auto compaction configuration*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+#### Sample request
+
+The following example creates an automatic compaction configuration for datasource `wikipedia_hour`. This automatic compaction configuration performs compaction on `wikipedia_hour`, resulting in compacted segments that represent a day interval of data.
+
+* `wikipedia_hour` is a datasource with `HOUR` segment granularity.
+* `skipOffsetFromLatest` is set to `PT0S`, meaning that no data is skipped.
+* `partitionsSpec` is set to the default `dynamic`, allowing Druid to dynamically determine the optimal partitioning strategy.
+* `type` is set to `index_parallel`, meaning that parallel indexing will be used.
+* `segmentGranularity` is set to `DAY`, meaning that each compacted segment will be a day of data.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"\
+--header 'Content-Type: application/json' \
+--data '{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}'
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+Content-Type: application/json
+Content-Length: 281
+
+{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
+
+
+### Remove datasource automatic compaction configuration
+
+Removes the automatic compaction configuration for a datasource. This updates the compaction status of the datasource to "Not enabled."
+
+#### URL
+
+<code class="deleteAPI">DELETE</code> <code>/druid/coordinator/v1/config/compaction/:dataSource</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully deleted automatic compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Datasource does not have automatic compaction or invalid datasource name*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request DELETE "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+DELETE /druid/coordinator/v1/config/compaction/wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
## Automatic compaction configuration
-`GET /druid/coordinator/v1/config/compaction`
+### Get all automatic compaction configuration
+
+Retrieves all automatic compaction configurations. Returns a `compactionConfigs` object containing the active automatic compaction configuration of all datasources.
+
+You can use this endpoint to retrieve `compactionTaskSlotRatio` and `maxCompactionTaskSlots` values for managing resource allocation of compaction tasks.
+
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved automatic compaction configurations*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
-Returns all automatic compaction configs.
+```json
+{
+ "compactionConfigs": [
+ {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ {
+ "dataSource": "wikipedia",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ }
+ ],
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+}
+```
+</details>
-`GET /druid/coordinator/v1/config/compaction/{dataSource}`
+### Get datasource automatic compaction configuration
-Returns an automatic compaction config of a dataSource.
+Retrieves the automatic compaction configuration of a datasource.
-`GET /druid/coordinator/v1/config/compaction/{dataSource}/history?interval={interval}&count={count}`
+#### URL
-Returns the history of the automatic compaction config for a dataSource. Optionally accepts `interval` and `count`
-query string parameters to filter by interval and limit the number of results respectively. If the dataSource does not
-exist or there is no compaction history for the dataSource, an empty list is returned.
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction/:dataSource</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved configuration for datasource*
+
+<!--404 NOT FOUND-->
+
+*Invalid datasource or datasource does not have automatic compaction enabled*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+The following example retrieves the automatic compaction configuration for datasource `wikipedia_hour`.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction/wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
+
+```json
+{
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+}
+```
+</details>
+
+### Get datasource automatic compaction configuration history
+
+Retrieves the history of the automatic compaction config for a datasource. Returns an empty list if the datasource does not exist or there is no compaction history for the datasource.
The response contains a list of objects with the following keys:
-* `globalConfig`: A json object containing automatic compaction config that applies to the entire cluster.
-* `compactionConfig`: A json object containing the automatic compaction config for the datasource.
-* `auditInfo`: A json object that contains information about the change made - like `author`, `comment` and `ip`.
+* `globalConfig`: A JSON object containing automatic compaction config that applies to the entire cluster.
+* `compactionConfig`: A JSON object containing the automatic compaction config for the datasource.
+* `auditInfo`: A JSON object that contains information about the change made - like `author`, `comment` and `ip`.
* `auditTime`: The date and time when the change was made.
-`POST /druid/coordinator/v1/config/compaction/taskslots?ratio={someRatio}&max={someMaxSlots}`
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction/:dataSource/history</code>
+
+#### Query parameters
+* `interval` (optional)
+ * Type: ISO-8601
+ * Limit the results within a specified interval. Use `/` as the delimiter for the interval string.
+* `count` (optional)
+ * Type: Int
+ * Limit the number of results.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved configuration history*
+
+<!--400 BAD REQUEST-->
+
+*Invalid `count` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour/history"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction/wikipedia_hour/history HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
+
+```json
+[
+ {
+ "globalConfig": {
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+ },
+ "compactionConfig": {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "P1D",
+ "tuningConfig": null,
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ "auditInfo": {
+ "author": "",
+ "comment": "",
+ "ip": "127.0.0.1"
+ },
+ "auditTime": "2023-07-31T18:15:19.302Z"
+ },
+ {
+ "globalConfig": {
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+ },
+ "compactionConfig": {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ "auditInfo": {
+ "author": "",
+ "comment": "",
+ "ip": "127.0.0.1"
+ },
+ "auditTime": "2023-07-31T18:16:16.362Z"
+ }
+]
+```
+</details>
+
+## Automatic compaction status
+
+### Get segments awaiting compaction
+
+Returns the total size of segments awaiting compaction for the given datasource. The specified datasource must have automatic compaction enabled.
+
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/compaction/progress?dataSource=:dataSource</code>
+
+#### Query parameter
+* `dataSource` (required)
+ * Type: String
+ * Name of the datasource for this status information
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved segment size awaiting compaction*
+
+<!--404 NOT FOUND-->
+
+*Unknown datasource name or datasource does not have automatic compaction enabled*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+The following example retrieves the remaining segments to be compacted for datasource `wikipedia_hour`.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/compaction/progress?dataSource=wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/compaction/progress?dataSource=wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
+
+```json
+{
+ "remainingSegmentSize": 7615837
+}
+```
+</details>
+
+
+### Get compaction status and statistics
+
+Retrieves the status and statistics from the automatic compaction run of all datasources which have automatic compaction enabled in the latest run. Returns a list of `latestStatus` objects. Each `latestStatus` represents the status for a datasource with automatic compaction enabled.
+
+The `latestStatus` object has the following keys:
Review Comment:
```suggestion
The `latestStatus` object has the following properties:
```
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,762 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. Automatic compaction can be configured in the Druid web console or API.
-## Automatic compaction status
+In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments.
-`GET /druid/coordinator/v1/compaction/progress?dataSource={dataSource}`
+## Manage automatic compaction
-Returns the total size of segments awaiting compaction for the given dataSource. The specified dataSource must have [automatic compaction](../data-management/automatic-compaction.md) enabled.
+### Update capacity for compaction tasks
-`GET /druid/coordinator/v1/compaction/status`
+Update the capacity for compaction tasks. Maximum number of compaction tasks is limited by `ratio` and `max`.
+`ratio` is ratio of the total task slots to the compaction task slots and the `max` is the number of task slots for compaction tasks.
-Returns the status and statistics from the auto-compaction run of all dataSources which have auto-compaction enabled in the latest run. The response payload includes a list of `latestStatus` objects. Each `latestStatus` represents the status for a dataSource (which has/had auto-compaction enabled).
+The minimum number of compaction tasks is 1 and the maximum is 2147483652.
-The `latestStatus` object has the following keys:
-* `dataSource`: name of the datasource for this status information
-* `scheduleStatus`: auto-compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active auto-compaction config submitted. Otherwise, returns `NOT_ENABLED`.
-* `bytesAwaitingCompaction`: total bytes of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `bytesCompacted`: total bytes of this datasource that are already compacted with the spec set in the auto-compaction config
-* `bytesSkipped`: total bytes of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `segmentCountAwaitingCompaction`: total number of segments of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `segmentCountCompacted`: total number of segments of this datasource that are already compacted with the spec set in the auto-compaction config
-* `segmentCountSkipped`: total number of segments of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `intervalCountAwaitingCompaction`: total number of intervals of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `intervalCountCompacted`: total number of intervals of this datasource that are already compacted with the spec set in the auto-compaction config
-* `intervalCountSkipped`: total number of intervals of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-
-`GET /druid/coordinator/v1/compaction/status?dataSource={dataSource}`
-
-Similar to the API `/druid/coordinator/v1/compaction/status` above but filters response to only return information for the dataSource given.
-The dataSource must have auto-compaction enabled.
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction/taskslots</code>
+
+#### Query parameters
+* `ratio` (optional)
+ * Type: Float
+ * Default: 0.1
+ * Limit the ratio of the total task slots to compaction task slots.
+* `max` (optional)
+ * Type: Int
+ * Default: 2147483647
+ * Limit the maximum number of task slots for compaction tasks.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully updated compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Invalid `max` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000"
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000 HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
+
+### Create or update datasource automatic compaction configuration
+
+Creates or updates the automatic compaction config for a datasource. The automatic compaction can be submitted as a JSON object in the request body.
+
+Automatic compaction configuration require only the `dataSource` property. All others will be filled with default values if not specified. See [Automatic compaction dynamic configuration](../configuration/index.md#automatic-compaction-dynamic-configuration) for configuration details.
+
+Note that this endpoint will return an HTTP `200 OK` even if the datasource name does not exist.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully submitted auto compaction configuration*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+#### Sample request
+
+The following example creates an automatic compaction configuration for datasource `wikipedia_hour`. This automatic compaction configuration performs compaction on `wikipedia_hour`, resulting in compacted segments that represent a day interval of data.
+
+* `wikipedia_hour` is a datasource with `HOUR` segment granularity.
+* `skipOffsetFromLatest` is set to `PT0S`, meaning that no data is skipped.
+* `partitionsSpec` is set to the default `dynamic`, allowing Druid to dynamically determine the optimal partitioning strategy.
+* `type` is set to `index_parallel`, meaning that parallel indexing will be used.
+* `segmentGranularity` is set to `DAY`, meaning that each compacted segment will be a day of data.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"\
+--header 'Content-Type: application/json' \
+--data '{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}'
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+Content-Type: application/json
+Content-Length: 281
+
+{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
+
+
+### Remove datasource automatic compaction configuration
+
+Removes the automatic compaction configuration for a datasource. This updates the compaction status of the datasource to "Not enabled."
+
+#### URL
+
+<code class="deleteAPI">DELETE</code> <code>/druid/coordinator/v1/config/compaction/:dataSource</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully deleted automatic compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Datasource does not have automatic compaction or invalid datasource name*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request DELETE "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+DELETE /druid/coordinator/v1/config/compaction/wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` and an empty response body.
## Automatic compaction configuration
-`GET /druid/coordinator/v1/config/compaction`
+### Get all automatic compaction configuration
+
+Retrieves all automatic compaction configurations. Returns a `compactionConfigs` object containing the active automatic compaction configuration of all datasources.
+
+You can use this endpoint to retrieve `compactionTaskSlotRatio` and `maxCompactionTaskSlots` values for managing resource allocation of compaction tasks.
+
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved automatic compaction configurations*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
-Returns all automatic compaction configs.
+```json
+{
+ "compactionConfigs": [
+ {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ {
+ "dataSource": "wikipedia",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ }
+ ],
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+}
+```
+</details>
-`GET /druid/coordinator/v1/config/compaction/{dataSource}`
+### Get datasource automatic compaction configuration
-Returns an automatic compaction config of a dataSource.
+Retrieves the automatic compaction configuration of a datasource.
-`GET /druid/coordinator/v1/config/compaction/{dataSource}/history?interval={interval}&count={count}`
+#### URL
-Returns the history of the automatic compaction config for a dataSource. Optionally accepts `interval` and `count`
-query string parameters to filter by interval and limit the number of results respectively. If the dataSource does not
-exist or there is no compaction history for the dataSource, an empty list is returned.
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction/:dataSource</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved configuration for datasource*
+
+<!--404 NOT FOUND-->
+
+*Invalid datasource or datasource does not have automatic compaction enabled*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+The following example retrieves the automatic compaction configuration for datasource `wikipedia_hour`.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction/wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
+
+```json
+{
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+}
+```
+</details>
+
+### Get datasource automatic compaction configuration history
+
+Retrieves the history of the automatic compaction config for a datasource. Returns an empty list if the datasource does not exist or there is no compaction history for the datasource.
The response contains a list of objects with the following keys:
-* `globalConfig`: A json object containing automatic compaction config that applies to the entire cluster.
-* `compactionConfig`: A json object containing the automatic compaction config for the datasource.
-* `auditInfo`: A json object that contains information about the change made - like `author`, `comment` and `ip`.
+* `globalConfig`: A JSON object containing automatic compaction config that applies to the entire cluster.
+* `compactionConfig`: A JSON object containing the automatic compaction config for the datasource.
+* `auditInfo`: A JSON object that contains information about the change made - like `author`, `comment` and `ip`.
* `auditTime`: The date and time when the change was made.
-`POST /druid/coordinator/v1/config/compaction/taskslots?ratio={someRatio}&max={someMaxSlots}`
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction/:dataSource/history</code>
+
+#### Query parameters
+* `interval` (optional)
+ * Type: ISO-8601
+ * Limit the results within a specified interval. Use `/` as the delimiter for the interval string.
+* `count` (optional)
+ * Type: Int
+ * Limit the number of results.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved configuration history*
+
+<!--400 BAD REQUEST-->
+
+*Invalid `count` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour/history"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction/wikipedia_hour/history HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
+
+```json
+[
+ {
+ "globalConfig": {
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+ },
+ "compactionConfig": {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "P1D",
+ "tuningConfig": null,
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ "auditInfo": {
+ "author": "",
+ "comment": "",
+ "ip": "127.0.0.1"
+ },
+ "auditTime": "2023-07-31T18:15:19.302Z"
+ },
+ {
+ "globalConfig": {
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+ },
+ "compactionConfig": {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ "auditInfo": {
+ "author": "",
+ "comment": "",
+ "ip": "127.0.0.1"
+ },
+ "auditTime": "2023-07-31T18:16:16.362Z"
+ }
+]
+```
+</details>
+
+## Automatic compaction status
+
+### Get segments awaiting compaction
+
+Returns the total size of segments awaiting compaction for the given datasource. The specified datasource must have automatic compaction enabled.
+
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/compaction/progress?dataSource=:dataSource</code>
+
+#### Query parameter
+* `dataSource` (required)
+ * Type: String
+ * Name of the datasource for this status information
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved segment size awaiting compaction*
+
+<!--404 NOT FOUND-->
+
+*Unknown datasource name or datasource does not have automatic compaction enabled*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+The following example retrieves the remaining segments to be compacted for datasource `wikipedia_hour`.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/compaction/progress?dataSource=wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/compaction/progress?dataSource=wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
+
+```json
+{
+ "remainingSegmentSize": 7615837
+}
+```
+</details>
+
+
+### Get compaction status and statistics
+
+Retrieves the status and statistics from the automatic compaction run of all datasources which have automatic compaction enabled in the latest run. Returns a list of `latestStatus` objects. Each `latestStatus` represents the status for a datasource with automatic compaction enabled.
Review Comment:
```suggestion
Retrieves an array of `latestStatus` objects representing the status and statistics from the latest automatic compaction run for all datasources with automatic compaction enabled.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] Automatic compaction API documentation refactor (druid)
Posted by "ektravel (via GitHub)" <gi...@apache.org>.
ektravel commented on code in PR #14740:
URL: https://github.com/apache/druid/pull/14740#discussion_r1290631691
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,762 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This topic describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. You can configure automatic compaction in the Druid web console or API.
-## Automatic compaction status
+In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use `http://localhost:8888` for quickstart deployments.
+
+## Manage automatic compaction
+
+### Update capacity for compaction tasks
+
+Update the capacity for compaction tasks. The minimum number of compaction tasks is 1 and the maximum is 2147483652.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction/taskslots</code>
+
+#### Query parameters
+
+To limit the maximum number of compaction tasks, use the optional query parameters `ratio` and `max`:
+
+* `ratio` (optional)
+ * Type: Float
+ * Default: 0.1
+ * Limit the ratio of the total task slots to compaction task slots.
+* `max` (optional)
+ * Type: Int
+ * Default: 2147483647
+ * Limit the maximum number of task slots for compaction tasks.
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully updated compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Invalid `max` value*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000"
+```
+
+<!--HTTP-->
+
+```HTTP
+POST /druid/coordinator/v1/config/compaction/taskslots?ratio=0.2&max=250000 HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` message code and an empty response body.
+
+### Create or update automatic compaction configuration
+
+Creates or updates the automatic compaction configuration for a datasource. You pass the automatic compaction as a JSON object in the request body.
+
+Automatic compaction configuration require only the `dataSource` property. Druid fills all other properties with default values if not specified. See [Automatic compaction dynamic configuration](../configuration/index.md#automatic-compaction-dynamic-configuration) for configuration details.
+
+Note that this endpoint returns an HTTP `200 OK` message code even if the datasource name does not exist.
+
+#### URL
+
+<code class="postAPI">POST</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully submitted auto compaction configuration*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+#### Sample request
+
+The following example creates an automatic compaction configuration for datasource `wikipedia_hour`. This automatic compaction configuration performs compaction on `wikipedia_hour`, resulting in compacted segments that represent a day interval of data. In this example:
+
+* `wikipedia_hour` is a datasource with `HOUR` segment granularity.
+* `skipOffsetFromLatest` is set to `PT0S`, meaning that no data is skipped.
+* `partitionsSpec` is set to the default `dynamic`, allowing Druid to dynamically determine the optimal partitioning strategy.
+* `type` is set to `index_parallel`, meaning that parallel indexing is used.
+* `segmentGranularity` is set to `DAY`, meaning that each compacted segment is a day of data.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
-`GET /druid/coordinator/v1/compaction/progress?dataSource={dataSource}`
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"\
+--header 'Content-Type: application/json' \
+--data '{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}'
+```
-Returns the total size of segments awaiting compaction for the given dataSource. The specified dataSource must have [automatic compaction](../data-management/automatic-compaction.md) enabled.
+<!--HTTP-->
-`GET /druid/coordinator/v1/compaction/status`
+```HTTP
+POST /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+Content-Type: application/json
+Content-Length: 281
-Returns the status and statistics from the auto-compaction run of all dataSources which have auto-compaction enabled in the latest run. The response payload includes a list of `latestStatus` objects. Each `latestStatus` represents the status for a dataSource (which has/had auto-compaction enabled).
+{
+ "dataSource": "wikipedia_hour",
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "partitionsSpec": {
+ "type": "dynamic"
+ },
+ "type": "index_parallel"
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY"
+ }
+}
+```
-The `latestStatus` object has the following keys:
-* `dataSource`: name of the datasource for this status information
-* `scheduleStatus`: auto-compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING`. Returns `RUNNING ` if the dataSource has an active auto-compaction config submitted. Otherwise, returns `NOT_ENABLED`.
-* `bytesAwaitingCompaction`: total bytes of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `bytesCompacted`: total bytes of this datasource that are already compacted with the spec set in the auto-compaction config
-* `bytesSkipped`: total bytes of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `segmentCountAwaitingCompaction`: total number of segments of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `segmentCountCompacted`: total number of segments of this datasource that are already compacted with the spec set in the auto-compaction config
-* `segmentCountSkipped`: total number of segments of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
-* `intervalCountAwaitingCompaction`: total number of intervals of this datasource waiting to be compacted by the auto-compaction (only consider intervals/segments that are eligible for auto-compaction)
-* `intervalCountCompacted`: total number of intervals of this datasource that are already compacted with the spec set in the auto-compaction config
-* `intervalCountSkipped`: total number of intervals of this datasource that are skipped (not eligible for auto-compaction) by the auto-compaction
+<!--END_DOCUSAURUS_CODE_TABS-->
-`GET /druid/coordinator/v1/compaction/status?dataSource={dataSource}`
+#### Sample response
-Similar to the API `/druid/coordinator/v1/compaction/status` above but filters response to only return information for the dataSource given.
-The dataSource must have auto-compaction enabled.
+A successful request returns an HTTP `200 OK` message code and an empty response body.
+
+
+### Remove automatic compaction configuration
+
+Removes the automatic compaction configuration for a datasource. This updates the compaction status of the datasource to "Not enabled."
+
+#### URL
+
+<code class="deleteAPI">DELETE</code> <code>/druid/coordinator/v1/config/compaction/:dataSource</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully deleted automatic compaction configuration*
+
+<!--404 NOT FOUND-->
+
+*Datasource does not have automatic compaction or invalid datasource name*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl --request DELETE "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+DELETE /druid/coordinator/v1/config/compaction/wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+A successful request returns an HTTP `200 OK` message code and an empty response body.
## Automatic compaction configuration
-`GET /druid/coordinator/v1/config/compaction`
+### Get all automatic compaction configurations
+
+Retrieves all automatic compaction configurations. Returns a `compactionConfigs` object containing the active automatic compaction configuration of all datasources.
+
+You can use this endpoint to retrieve `compactionTaskSlotRatio` and `maxCompactionTaskSlots` values for managing resource allocation of compaction tasks.
+
+#### URL
+
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved automatic compaction configurations*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
-Returns all automatic compaction configs.
+```json
+{
+ "compactionConfigs": [
+ {
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ },
+ {
+ "dataSource": "wikipedia",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+ }
+ ],
+ "compactionTaskSlotRatio": 0.1,
+ "maxCompactionTaskSlots": 2147483647,
+ "useAutoScaleSlots": false
+}
+```
+</details>
-`GET /druid/coordinator/v1/config/compaction/{dataSource}`
+### Get automatic compaction configuration
-Returns an automatic compaction config of a dataSource.
+Retrieves the automatic compaction configuration of a datasource.
-`GET /druid/coordinator/v1/config/compaction/{dataSource}/history?interval={interval}&count={count}`
+#### URL
-Returns the history of the automatic compaction config for a dataSource. Optionally accepts `interval` and `count`
-query string parameters to filter by interval and limit the number of results respectively. If the dataSource does not
-exist or there is no compaction history for the dataSource, an empty list is returned.
+<code class="getAPI">GET</code> <code>/druid/coordinator/v1/config/compaction/:dataSource</code>
+
+#### Responses
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--200 SUCCESS-->
+
+*Successfully retrieved configuration for datasource*
+
+<!--404 NOT FOUND-->
+
+*Invalid datasource or datasource does not have automatic compaction enabled*
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+---
+
+#### Sample request
+
+The following example retrieves the automatic compaction configuration for datasource `wikipedia_hour`.
+
+<!--DOCUSAURUS_CODE_TABS-->
+
+<!--cURL-->
+
+```shell
+curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/compaction/wikipedia_hour"
+```
+
+<!--HTTP-->
+
+```HTTP
+GET /druid/coordinator/v1/config/compaction/wikipedia_hour HTTP/1.1
+Host: http://ROUTER_IP:ROUTER_PORT
+```
+
+<!--END_DOCUSAURUS_CODE_TABS-->
+
+#### Sample response
+
+<details>
+ <summary>Click to show sample response</summary>
+
+```json
+{
+ "dataSource": "wikipedia_hour",
+ "taskPriority": 25,
+ "inputSegmentSizeBytes": 100000000000000,
+ "maxRowsPerSegment": null,
+ "skipOffsetFromLatest": "PT0S",
+ "tuningConfig": {
+ "maxRowsInMemory": null,
+ "appendableIndexSpec": null,
+ "maxBytesInMemory": null,
+ "maxTotalRows": null,
+ "splitHintSpec": null,
+ "partitionsSpec": {
+ "type": "dynamic",
+ "maxRowsPerSegment": 5000000,
+ "maxTotalRows": null
+ },
+ "indexSpec": null,
+ "indexSpecForIntermediatePersists": null,
+ "maxPendingPersists": null,
+ "pushTimeout": null,
+ "segmentWriteOutMediumFactory": null,
+ "maxNumConcurrentSubTasks": null,
+ "maxRetry": null,
+ "taskStatusCheckPeriodMs": null,
+ "chatHandlerTimeout": null,
+ "chatHandlerNumRetries": null,
+ "maxNumSegmentsToMerge": null,
+ "totalNumMergeTasks": null,
+ "maxColumnsToMerge": null,
+ "type": "index_parallel",
+ "forceGuaranteedRollup": false
+ },
+ "granularitySpec": {
+ "segmentGranularity": "DAY",
+ "queryGranularity": null,
+ "rollup": null
+ },
+ "dimensionsSpec": null,
+ "metricsSpec": null,
+ "transformSpec": null,
+ "ioConfig": null,
+ "taskContext": null
+}
+```
+</details>
+
+### Get automatic compaction configuration history
+
+Retrieves the history of the automatic compaction configuration for a datasource. Returns an empty list if the datasource does not exist or there is no compaction history for the datasource.
The response contains a list of objects with the following keys:
-* `globalConfig`: A json object containing automatic compaction config that applies to the entire cluster.
-* `compactionConfig`: A json object containing the automatic compaction config for the datasource.
-* `auditInfo`: A json object that contains information about the change made - like `author`, `comment` and `ip`.
+* `globalConfig`: A JSON object containing automatic compaction configuration that applies to the entire cluster.
+* `compactionConfig`: A JSON object containing the automatic compaction configuration for the datasource.
+* `auditInfo`: A JSON object that contains information about the change made, such as `author`, `comment` or `ip`.
Review Comment:
```suggestion
* `auditInfo`: A JSON object containing information about the change made, such as `author`, `comment` or `ip`.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] Automatic compaction API documentation refactor (druid)
Posted by "ektravel (via GitHub)" <gi...@apache.org>.
ektravel commented on code in PR #14740:
URL: https://github.com/apache/druid/pull/14740#discussion_r1290610506
##########
docs/api-reference/automatic-compaction-api.md:
##########
@@ -23,69 +23,762 @@ sidebar_label: Automatic compaction
~ under the License.
-->
-This document describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid.
+This topic describes status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. You can configure automatic compaction in the Druid web console or API.
Review Comment:
```suggestion
This topic describes the status and configuration API endpoints for [automatic compaction](../data-management/automatic-compaction.md) in Apache Druid. You can configure automatic compaction in the Druid web console or API.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org