You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by vi...@apache.org on 2023/05/18 00:05:41 UTC
[druid] branch master updated: Fixup data deletion tutorial docs (#14283)
This is an automated email from the ASF dual-hosted git repository.
victoria pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/druid.git
The following commit(s) were added to refs/heads/master by this push:
new 7400ed3c93 Fixup data deletion tutorial docs (#14283)
7400ed3c93 is described below
commit 7400ed3c9325e9237c52ea12a5fcf490e603b249
Author: Abhishek Radhakrishnan <ab...@gmail.com>
AuthorDate: Wed May 17 17:05:35 2023 -0700
Fixup data deletion tutorial docs (#14283)
Co-authored-by: Victoria Lim <vt...@users.noreply.github.com>
---
docs/assets/tutorial-deletion-01.png | Bin 43586 -> 163245 bytes
docs/assets/tutorial-deletion-02.png | Bin 439602 -> 159346 bytes
docs/assets/tutorial-deletion-03.png | Bin 437304 -> 153770 bytes
docs/operations/api-reference.md | 6 +-
docs/tutorials/tutorial-delete-data.md | 61 ++++++++++++---------
.../tutorial/deletion-disable-segments.json | 7 ---
6 files changed, 37 insertions(+), 37 deletions(-)
diff --git a/docs/assets/tutorial-deletion-01.png b/docs/assets/tutorial-deletion-01.png
index de68d381ca..942f057d7e 100644
Binary files a/docs/assets/tutorial-deletion-01.png and b/docs/assets/tutorial-deletion-01.png differ
diff --git a/docs/assets/tutorial-deletion-02.png b/docs/assets/tutorial-deletion-02.png
index ffe4585292..516fdf7fe8 100644
Binary files a/docs/assets/tutorial-deletion-02.png and b/docs/assets/tutorial-deletion-02.png differ
diff --git a/docs/assets/tutorial-deletion-03.png b/docs/assets/tutorial-deletion-03.png
index 221774f18e..666ff7a89e 100644
Binary files a/docs/assets/tutorial-deletion-03.png and b/docs/assets/tutorial-deletion-03.png differ
diff --git a/docs/operations/api-reference.md b/docs/operations/api-reference.md
index babc74f9c1..af390e0774 100644
--- a/docs/operations/api-reference.md
+++ b/docs/operations/api-reference.md
@@ -293,10 +293,10 @@ Returns full segment metadata for a specific segment in the cluster.
Return the tiers that a datasource exists in.
-#### Note for coordinator's POST and DELETE API's
-The segments would be enabled when these API's are called, but then can be disabled again by the coordinator if any dropRule matches. Segments enabled by these API's might not be loaded by historical processes if no loadRule matches. If an indexing or kill task runs at the same time as these API's are invoked, the behavior is undefined. Some segments might be killed and others might be enabled. It's also possible that all segments might be disabled but at the same time, the indexing tas [...]
+#### Note for Coordinator's POST and DELETE APIs
+While segments may be enabled by issuing POST requests for the datasources, the Coordinator may again disable segments if they match any configured [drop rules](../operations/rule-configuration.md#drop-rules). Even if segments are enabled by these APIs, you must configure a [load rule](../operations/rule-configuration.md#load-rules) to load them onto Historical processes. If an indexing or kill task runs at the same time these APIs are invoked, the behavior is undefined. Some segments mi [...]
-> Caution : Avoid using indexing or kill tasks and these API's at the same time for the same datasource and time chunk. (It's fine if the time chunks or datasource don't overlap)
+> Avoid using indexing or kill tasks and these APIs at the same time for the same datasource and time chunk.
`POST /druid/coordinator/v1/datasources/{dataSourceName}`
diff --git a/docs/tutorials/tutorial-delete-data.md b/docs/tutorials/tutorial-delete-data.md
index ba2a6f3ee8..93173470c4 100644
--- a/docs/tutorials/tutorial-delete-data.md
+++ b/docs/tutorials/tutorial-delete-data.md
@@ -26,8 +26,9 @@ sidebar_label: "Deleting data"
This tutorial demonstrates how to delete existing data.
-For this tutorial, we'll assume you've already downloaded Apache Druid as described in
-the [single-machine quickstart](index.md) and have it running on your local machine.
+This tutorial requires the following:
+* A running Apache Druid instance. If you don't have Druid, see the [single-machine quickstart](index.md) to get started.
+* The command-line JSON processor, [jq](https://stedolan.github.io/jq/download/).
## Load initial data
@@ -59,9 +60,8 @@ Let's disable segments in interval `2015-09-12T18:00:00.000Z/2015-09-12T20:00:00
curl -X 'POST' -H 'Content-Type:application/json' -d '{ "interval" : "2015-09-12T18:00:00.000Z/2015-09-12T20:00:00.000Z" }' http://localhost:8081/druid/coordinator/v1/datasources/deletion-tutorial/markUnused
```
-After that command completes, you should see that the segment for hour 18 and 19 have been disabled:
-
-
+When the request completes, the Segments view of the web console no longer displays the segments for hours 18 and 19.
+
Note that the hour 18 and 19 segments are still present in deep storage:
@@ -95,37 +95,31 @@ $ ls -l1 var/druid/segments/deletion-tutorial/
## Disable segments by segment IDs
-Let's disable some segments by their segmentID. This will again mark the segments as "unused", but not remove them from deep storage. You can see the full segmentID for a segment from UI as explained below.
-
-In the [segments view](http://localhost:8888/unified-console.html#segments), click the arrow on the left side of one of the remaining segments to expand the segment entry:
+Let's disable some segments by their segmentID. This will again mark the segments as "unused", but not remove them from deep storage. You can see the full segmentID for a segment using the web console.
-
+In the [segments view](http://localhost:8888/unified-console.html#segments), click one of the segment rows to open the segment metadata dialog:
-The top of the info box shows the full segment ID, e.g. `deletion-tutorial_2015-09-12T14:00:00.000Z_2015-09-12T15:00:00.000Z_2019-02-28T01:11:51.606Z` for the segment of hour 14.
+
-Let's disable the hour 13 and 14 segments by sending a POST request to the Coordinator with this payload
+The `identifier` field in the metadata dialog shows the full segment ID. For example, the hour 23 segment has segment ID `deletion-tutorial_2015-09-12T23:00:00.000Z_2015-09-13T00:00:00.000Z_2023-05-16T00:04:12.091Z`.
-```json
-{
- "segmentIds":
- [
- "deletion-tutorial_2015-09-12T13:00:00.000Z_2015-09-12T14:00:00.000Z_2019-05-01T17:38:46.961Z",
- "deletion-tutorial_2015-09-12T14:00:00.000Z_2015-09-12T15:00:00.000Z_2019-05-01T17:38:46.961Z"
- ]
-}
+Disable the last two segments, hour 22 and 23 segments, by sending a POST request to the Coordinator with the corresponding segment IDs.
+The following command queries the Coordinator for segment IDs and uses `jq` to parse and extract the IDs of the last two segments.
+The segment IDs are stored in an environment variable named `unusedSegmentIds`.
+```bash
+unusedSegmentIds=$(curl -X 'GET' -H 'Content-Type:application/json' http://localhost:8081/druid/coordinator/v1/datasources/deletion-tutorial/segments | jq '.[-2:]')
```
-This payload json has been provided at `quickstart/tutorial/deletion-disable-segments.json`. Submit the POST request to Coordinator like this:
-
+The following request marks the segments unused:
```bash
-curl -X 'POST' -H 'Content-Type:application/json' -d @quickstart/tutorial/deletion-disable-segments.json http://localhost:8081/druid/coordinator/v1/datasources/deletion-tutorial/markUnused
+curl -X 'POST' -H 'Content-Type:application/json' -d "{\"segmentIds\": $unusedSegmentIds}" http://localhost:8081/druid/coordinator/v1/datasources/deletion-tutorial/markUnused
```
-After that command completes, you should see that the segments for hour 13 and 14 have been disabled:
+When the request completes, the Segments view of the web console no longer displays the segments for hours 22 and 23.
-Note that the hour 13 and 14 segments are still in deep storage:
+Note that the hour 22 and 23 segments are still in deep storage:
```bash
$ ls -l1 var/druid/segments/deletion-tutorial/
@@ -165,16 +159,29 @@ A Kill Task spec has been provided at `quickstart/tutorial/deletion-kill.json`.
curl -X 'POST' -H 'Content-Type:application/json' -d @quickstart/tutorial/deletion-kill.json http://localhost:8081/druid/indexer/v1/task
```
-After this task completes, you can see that the disabled segments have now been removed from deep storage:
+When the task finishes, note that Druid deleted the disabled segments from deep storage.
+
```bash
$ ls -l1 var/druid/segments/deletion-tutorial/
+2015-09-12T00:00:00.000Z_2015-09-12T01:00:00.000Z
+2015-09-12T01:00:00.000Z_2015-09-12T02:00:00.000Z
+2015-09-12T02:00:00.000Z_2015-09-12T03:00:00.000Z
+2015-09-12T03:00:00.000Z_2015-09-12T04:00:00.000Z
+2015-09-12T04:00:00.000Z_2015-09-12T05:00:00.000Z
+2015-09-12T05:00:00.000Z_2015-09-12T06:00:00.000Z
+2015-09-12T06:00:00.000Z_2015-09-12T07:00:00.000Z
+2015-09-12T07:00:00.000Z_2015-09-12T08:00:00.000Z
+2015-09-12T08:00:00.000Z_2015-09-12T09:00:00.000Z
+2015-09-12T09:00:00.000Z_2015-09-12T10:00:00.000Z
+2015-09-12T10:00:00.000Z_2015-09-12T11:00:00.000Z
+2015-09-12T11:00:00.000Z_2015-09-12T12:00:00.000Z
2015-09-12T12:00:00.000Z_2015-09-12T13:00:00.000Z
+2015-09-12T13:00:00.000Z_2015-09-12T14:00:00.000Z
+2015-09-12T14:00:00.000Z_2015-09-12T15:00:00.000Z
2015-09-12T15:00:00.000Z_2015-09-12T16:00:00.000Z
2015-09-12T16:00:00.000Z_2015-09-12T17:00:00.000Z
2015-09-12T17:00:00.000Z_2015-09-12T18:00:00.000Z
2015-09-12T20:00:00.000Z_2015-09-12T21:00:00.000Z
2015-09-12T21:00:00.000Z_2015-09-12T22:00:00.000Z
-2015-09-12T22:00:00.000Z_2015-09-12T23:00:00.000Z
-2015-09-12T23:00:00.000Z_2015-09-13T00:00:00.000Z
```
diff --git a/examples/quickstart/tutorial/deletion-disable-segments.json b/examples/quickstart/tutorial/deletion-disable-segments.json
deleted file mode 100644
index 920e071a61..0000000000
--- a/examples/quickstart/tutorial/deletion-disable-segments.json
+++ /dev/null
@@ -1,7 +0,0 @@
-{
- "segmentIds":
- [
- "deletion-tutorial_2015-09-12T13:00:00.000Z_2015-09-12T14:00:00.000Z_2019-05-01T17:38:46.961Z",
- "deletion-tutorial_2015-09-12T14:00:00.000Z_2015-09-12T15:00:00.000Z_2019-05-01T17:38:46.961Z"
- ]
-}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org