You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2022/10/14 18:33:30 UTC
[GitHub] [druid] 317brian opened a new pull request, #13228: docs(msq): update insert vs replace for dimension-based segment pruning
317brian opened a new pull request, #13228:
URL: https://github.com/apache/druid/pull/13228
### Description
Updates the docs to be more explicit
Release note: n/a
<hr>
<hr>
<!-- Check the items by putting "x" in the brackets for the done things. Not all of these items apply to every PR. Remove the items which are not done or not relevant to the PR. None of the items from the checklist below are strictly necessary, but it would be very helpful if you at least self-review the PR. -->
This PR has:
- [x] been self-reviewed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] kfaraz commented on a diff in pull request #13228: docs(msq): update insert vs replace for dimension-based segment pruning
Posted by GitBox <gi...@apache.org>.
kfaraz commented on code in PR #13228:
URL: https://github.com/apache/druid/pull/13228#discussion_r996247003
##########
docs/multi-stage-query/concepts.md:
##########
@@ -102,6 +104,19 @@ issues](./known-issues.md#select) page.
For more information about the syntax, see [REPLACE](./reference.md#replace).
+When deciding whether to use REPLACE or INSERT, keep in mind that REPLACE can perform dimension-based segment pruning but INSERT cannot. To use dimension-based segment pruning, your query needs to meet the following requirements:
Review Comment:
This part and the above change on line 81/82 needs to be rephrased.
To clarify, INSERT or REPLACE queries (better called statements for less ambiguity in this case?) cannot and need not perform dimension-based pruning (* unless they are doing a sub-SELECT on some other datasource).
It is the queries (i.e. SELECT queries of any kind) that we run on the _segments_ generated with the above INSERT or REPLACE statements that could potentially benefit from such pruning.
As a suggestion:
```suggestion
When deciding whether to use REPLACE or INSERT, keep in mind that segments generated with REPLACE can be pruned with dimension-based pruning but those generated with INSERT cannot. Please refer to [Clustering](#clustering) for the prerequisites of dimension-based pruning.:
```
##########
docs/multi-stage-query/concepts.md:
##########
@@ -102,6 +104,19 @@ issues](./known-issues.md#select) page.
For more information about the syntax, see [REPLACE](./reference.md#replace).
+When deciding whether to use REPLACE or INSERT, keep in mind that REPLACE can perform dimension-based segment pruning but INSERT cannot. To use dimension-based segment pruning, your query needs to meet the following requirements:
+
+- Segments were generated by a REPLACE statement, not an INSERT statement.
Review Comment:
I don't think this section needs to be reiterated here as this info is already captured later in the "Clustering" section. I think just linking to that section is enough.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] 317brian commented on a diff in pull request #13228: docs(msq): update insert vs replace for dimension-based segment pruning
Posted by GitBox <gi...@apache.org>.
317brian commented on code in PR #13228:
URL: https://github.com/apache/druid/pull/13228#discussion_r1007400355
##########
docs/multi-stage-query/concepts.md:
##########
@@ -102,6 +104,19 @@ issues](./known-issues.md#select) page.
For more information about the syntax, see [REPLACE](./reference.md#replace).
+When deciding whether to use REPLACE or INSERT, keep in mind that REPLACE can perform dimension-based segment pruning but INSERT cannot. To use dimension-based segment pruning, your query needs to meet the following requirements:
Review Comment:
Linking to it is probably a better solution than straight repeating. Will make this change
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] 317brian commented on pull request #13228: docs(msq): update insert vs replace for dimension-based segment pruning
Posted by GitBox <gi...@apache.org>.
317brian commented on PR #13228:
URL: https://github.com/apache/druid/pull/13228#issuecomment-1292278483
Thanks for the review @kfaraz ! I'll take a look at your comments this week. (I didn't get a notification email for them for some reason, so apologies for the late response.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] kfaraz commented on a diff in pull request #13228: docs(msq): update insert vs replace for dimension-based segment pruning
Posted by GitBox <gi...@apache.org>.
kfaraz commented on code in PR #13228:
URL: https://github.com/apache/druid/pull/13228#discussion_r996247003
##########
docs/multi-stage-query/concepts.md:
##########
@@ -102,6 +104,19 @@ issues](./known-issues.md#select) page.
For more information about the syntax, see [REPLACE](./reference.md#replace).
+When deciding whether to use REPLACE or INSERT, keep in mind that REPLACE can perform dimension-based segment pruning but INSERT cannot. To use dimension-based segment pruning, your query needs to meet the following requirements:
Review Comment:
This part and the above change on line 81/82 needs to be rephrased.
To clarify, INSERT or REPLACE queries (better called statements for less ambiguity in this case?) cannot and need not perform dimension-based pruning (* unless they include a sub-SELECT on some other datasource).
It is the queries (i.e. SELECT queries of any kind) that we run on the _segments_ generated with the above INSERT or REPLACE statements that could potentially benefit from such pruning.
As a suggestion:
```suggestion
When deciding whether to use REPLACE or INSERT, keep in mind that segments generated with REPLACE can be pruned with dimension-based pruning but those generated with INSERT cannot. Please refer to [Clustering](#clustering) for the prerequisites of dimension-based pruning.:
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] kfaraz merged pull request #13228: docs(msq): update insert vs replace for dimension-based segment pruning
Posted by GitBox <gi...@apache.org>.
kfaraz merged PR #13228:
URL: https://github.com/apache/druid/pull/13228
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org