You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by "317brian (via GitHub)" <gi...@apache.org> on 2023/10/20 18:16:40 UTC

[PR] docs: blurb about msq union all (druid)

317brian opened a new pull request, #15223:
URL: https://github.com/apache/druid/pull/15223

   
   ### Description
   Add a blurb about how MSQ treats UNION ALL queries
   
   <hr>
   
   
   This PR has:
   
   - [x] been self-reviewed.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org

Re: [PR] docs: blurb about msq union all (druid)

Posted by "LakshSingla (via GitHub)" <gi...@apache.org>.
LakshSingla commented on code in PR #15223:
URL: https://github.com/apache/druid/pull/15223#discussion_r1369086066


##########
docs/querying/sql.md:
##########
@@ -218,9 +218,10 @@ UNION ALL
 SELECT COUNT(*) FROM tbl WHERE my_column = 'value2'
 ```
 
-:::info
- With top-level queries, you can't apply GROUP BY, ORDER BY, or any other operator to the results of a UNION ALL.
-:::
+Keep the following in mind when writing top-level UNION ALL queries:
+
+- You can't apply GROUP BY, ORDER BY, or any other operator to the results of a UNION ALL.
+- If you use the MSQ task engine for the query, the SQL planner attempts to plan the top-level UNION ALL as a table-level UNION ALL. Because of this, the query behaves the same as a table-level UNION ALL, meaning it has the same characteristics and limitations. If the planner can't plan the query as a table-level UNION ALL, the query fails.

Review Comment:
   ```suggestion
   - If you use the MSQ task engine for the query, the SQL planner attempts to plan the top-level UNION ALL as a table-level UNION ALL. Therefore it has the same limitations as the table level union all - You can only union TableDataSources with simple projections - select columns and rename columns.  Clauses like GROUP BY, ORDER BY, etc are not supported on operands of union. 
   The query isn't supported if the planner can't plan it as a table-level union all
   ```



##########
docs/querying/sql.md:
##########
@@ -218,9 +218,10 @@ UNION ALL
 SELECT COUNT(*) FROM tbl WHERE my_column = 'value2'
 ```
 
-:::info
- With top-level queries, you can't apply GROUP BY, ORDER BY, or any other operator to the results of a UNION ALL.
-:::
+Keep the following in mind when writing top-level UNION ALL queries:
+
+- You can't apply GROUP BY, ORDER BY, or any other operator to the results of a UNION ALL.

Review Comment:
   Can coalesce into a single paragraph as mentioned in the other comment
   ```suggestion
   
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org

Re: [PR] docs: blurb about msq union all (druid)

Posted by "317brian (via GitHub)" <gi...@apache.org>.
317brian commented on code in PR #15223:
URL: https://github.com/apache/druid/pull/15223#discussion_r1372577129


##########
docs/querying/sql.md:
##########
@@ -218,9 +218,10 @@ UNION ALL
 SELECT COUNT(*) FROM tbl WHERE my_column = 'value2'
 ```
 
-:::info
- With top-level queries, you can't apply GROUP BY, ORDER BY, or any other operator to the results of a UNION ALL.
-:::
+Keep the following in mind when writing top-level UNION ALL queries:
+
+- You can't apply GROUP BY, ORDER BY, or any other operator to the results of a UNION ALL.

Review Comment:
   https://github.com/apache/druid/pull/15223/commits/9ab413099ce3a7a68f42cd75ba8b08ad29d702c7 has the new paragraph



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org

Re: [PR] docs: blurb about msq union all (druid)

Posted by "317brian (via GitHub)" <gi...@apache.org>.
317brian commented on code in PR #15223:
URL: https://github.com/apache/druid/pull/15223#discussion_r1369294278


##########
docs/querying/sql.md:
##########
@@ -218,9 +218,10 @@ UNION ALL
 SELECT COUNT(*) FROM tbl WHERE my_column = 'value2'
 ```
 
-:::info
- With top-level queries, you can't apply GROUP BY, ORDER BY, or any other operator to the results of a UNION ALL.
-:::
+Keep the following in mind when writing top-level UNION ALL queries:
+
+- You can't apply GROUP BY, ORDER BY, or any other operator to the results of a UNION ALL.

Review Comment:
   This limitation predates my changes. So does it no longer apply to non-MSQ queries?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org

Re: [PR] docs: blurb about msq union all (druid)

Posted by "LakshSingla (via GitHub)" <gi...@apache.org>.
LakshSingla commented on code in PR #15223:
URL: https://github.com/apache/druid/pull/15223#discussion_r1371180423


##########
docs/querying/sql.md:
##########
@@ -218,9 +218,10 @@ UNION ALL
 SELECT COUNT(*) FROM tbl WHERE my_column = 'value2'
 ```
 
-:::info
- With top-level queries, you can't apply GROUP BY, ORDER BY, or any other operator to the results of a UNION ALL.
-:::
+Keep the following in mind when writing top-level UNION ALL queries:
+
+- You can't apply GROUP BY, ORDER BY, or any other operator to the results of a UNION ALL.

Review Comment:
   This limitation is still present, but as per the previous comment, we can make it cohesive by coalescing both of the limitations into one big para. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org

Re: [PR] docs: blurb about msq union all (druid)

Posted by "ektravel (via GitHub)" <gi...@apache.org>.
ektravel commented on code in PR #15223:
URL: https://github.com/apache/druid/pull/15223#discussion_r1374663754


##########
docs/querying/sql.md:
##########
@@ -218,9 +218,7 @@ UNION ALL
 SELECT COUNT(*) FROM tbl WHERE my_column = 'value2'
 ```
 
-:::info
- With top-level queries, you can't apply GROUP BY, ORDER BY, or any other operator to the results of a UNION ALL.
-:::
+When you use a top-level UNION ALL, there are certain limitations that apply. For all top-level UNION ALL queries, you can't apply a GROUP BY, ORDER BY, or any other operator to the results of the query. For any top-level UNION ALL that uses the MSQ task engine, the SQL planner attempts to plan the top-level UNION ALL as a table-level UNION ALL. Because of this, UNION ALL queries that use the MSQ task engine always behave the same as a table-level UNION ALL queries. They have the same characteristics and limitations. If the planner can't plan the query as a table-level UNION ALL, the query fails.

Review Comment:
   ```suggestion
   Certain limitations apply when you use a top-level UNION ALL. For all top-level UNION ALL queries, you can't apply a GROUP BY, ORDER BY, or any other operator to the results of the query. For any top-level UNION ALL that uses the MSQ task engine, the SQL planner attempts to plan the top-level UNION ALL as a table-level UNION ALL. Because of this, UNION ALL queries that use the MSQ task engine always behave the same as table-level UNION ALL queries. They have the same characteristics and limitations. If the planner can't plan the query as a table-level UNION ALL, the query fails.
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org

Re: [PR] docs: blurb about msq union all (druid)

Posted by "317brian (via GitHub)" <gi...@apache.org>.
317brian commented on code in PR #15223:
URL: https://github.com/apache/druid/pull/15223#discussion_r1372088034


##########
docs/querying/sql.md:
##########
@@ -218,9 +218,10 @@ UNION ALL
 SELECT COUNT(*) FROM tbl WHERE my_column = 'value2'
 ```
 
-:::info
- With top-level queries, you can't apply GROUP BY, ORDER BY, or any other operator to the results of a UNION ALL.
-:::
+Keep the following in mind when writing top-level UNION ALL queries:
+
+- You can't apply GROUP BY, ORDER BY, or any other operator to the results of a UNION ALL.

Review Comment:
   Ok, if we want it in 1 paragraph, I'll need to rewrite the paragraph first. Starting with `If you use the MSQ task engine for the query,` sets the reader up to think that the whole paragraph is only about the MSQ task engine limitations



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org

Re: [PR] docs: blurb about msq union all (druid)

Posted by "317brian (via GitHub)" <gi...@apache.org>.
317brian commented on code in PR #15223:
URL: https://github.com/apache/druid/pull/15223#discussion_r1369294278


##########
docs/querying/sql.md:
##########
@@ -218,9 +218,10 @@ UNION ALL
 SELECT COUNT(*) FROM tbl WHERE my_column = 'value2'
 ```
 
-:::info
- With top-level queries, you can't apply GROUP BY, ORDER BY, or any other operator to the results of a UNION ALL.
-:::
+Keep the following in mind when writing top-level UNION ALL queries:
+
+- You can't apply GROUP BY, ORDER BY, or any other operator to the results of a UNION ALL.

Review Comment:
   This limitation predates my changes and seems to apply to any query engine. So does it no longer apply to non-MSQ queries?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org

Re: [PR] docs: blurb about msq union all (druid)

Posted by "317brian (via GitHub)" <gi...@apache.org>.
317brian merged PR #15223:
URL: https://github.com/apache/druid/pull/15223


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org