You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "michaelritsema (via GitHub)" <gi...@apache.org> on 2024/05/20 17:50:23 UTC
[I] [SIP] Proposal for aggregate dimensions [superset]
michaelritsema opened a new issue, #28610:
URL: https://github.com/apache/superset/issues/28610
*Please make sure you are familiar with the SIP process documented*
[here](https://github.com/apache/superset/issues/5602). The SIP will be numbered by a committer upon acceptance.
## [SIP] Proposal for aggregate dimensions
### Motivation
Adding columns to a report that are functionality dependent on an already added dimension should come at a low cost.
For example:
'GROUP BY person_id'
If we then want to add columns to the report "first_name,last_name,age" we'd have the choice of adding it to the dimension or as a metric.
Simply adding it to the dimension incurs penalty of putting in the GROUP BY. This is an expensive operation as well as a potential error (what if one of the rows actually had the wrong age and we ended up creating a new person)
Simply adding it as a metric is a bit awkward but is the best fit for my use case now. I have dozens of these columns and perfer to just wrap them in an ANY aggregate function.
### Proposed Change
Describe how the feature will be implemented, or the problem will be solved. If possible, include mocks, screenshots, or screencasts (even if from different tools).
I propose we add a toggle to set a dimension as an "aggregate dimension" Instead of adding this column to the group by the user can just pick the aggregate it would like (min,max,any, first, etc). By default we would not have to duplicate this as a new metric column but could keep the same name. I think this would also work with the newer Drill features in a better way than the metrics.
### New or Changed Public Interfaces
Describe any new additions to the model, views or `REST` endpoints. Describe any changes to existing visualizations, dashboards and React components. Describe changes that affect the Superset CLI and how Superset is deployed.
### Rejected Alternatives
Currently adding these strings as a metric in an ANY aggregate works best for me. This awkward and confusing to the end user to see dimensional type data as well as a lot of duplicated column naming "any(first_name) as first_name"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] [SIP-135] Proposal for aggregate dimensions [superset]
Posted by "michaelritsema (via GitHub)" <gi...@apache.org>.
michaelritsema commented on issue #28610:
URL: https://github.com/apache/superset/issues/28610#issuecomment-2147846348
If you had a denormalized table with a persons information called PersonSightings
with a row like:
person_id:1,f_name:evan,l_name:r, address:"343232 W. Palm St Jacksonville,Flordia",:sighting_timestamp:2024-01-01
imagine there was a new row for everytime a camera spotted the person
You would want to take advantage of the fact that most of the columsn are functionality depenendt on person_id
Currently to get a report on distinct sightings you'd need to add most of the columns as a dimension which generates:
SELECT person_id,f_name,l_name, address,sighting_timestamp,count(distinct sighting_timestamp)
from PersonSightings
GROUP BY person_id,f_name,l_name, address,sighting_timestamp
The main problem is the GROUP BY is very inefficient. I'd want SQL Like
SELECT person_id,any(f_name),any(l_name), any(address),any(sighting_timestamp),count(distinct sighting_timestamp)
from PersonSightings
GROUP BY person_id
To do this I need to turn those dimensions into metrics and add custom sql. But they should be treated as dimensions not metrics.
So I proposed a way to put an aggregate around a dimension so superset can apply ANY or FIRST (whatever the database supports)
In my use case I actually have a few dozen of these and using clickhouse which can be exponentially faster with smaller GROUP BYs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] [SIP-135] Proposal for aggregate dimensions [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on issue #28610:
URL: https://github.com/apache/superset/issues/28610#issuecomment-2176382378
@yousoph @geido - please check this out and let us know your thoughts :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] [SIP-135] Proposal for aggregate dimensions [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas closed issue #28610: [SIP-135] Proposal for aggregate dimensions
URL: https://github.com/apache/superset/issues/28610
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] [SIP-135] Proposal for aggregate dimensions [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on issue #28610:
URL: https://github.com/apache/superset/issues/28610#issuecomment-2147815409
I'm not quite understanding the proposal, to be honest. Can you provide some examples (both good and bad) of the use cases and the queries they'd generate?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] [SIP-135] Proposal for aggregate dimensions [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on issue #28610:
URL: https://github.com/apache/superset/issues/28610#issuecomment-2341410215
Closing for lack of follow-up clarity. I'm assuming this is a feature request, essentially. Please open a GitHub Discussion under the "Ideas" category if you want to seek help adding this. Happy to reopen this if you can provide more clarity of approach.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] [SIP-135] Proposal for aggregate dimensions [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on issue #28610:
URL: https://github.com/apache/superset/issues/28610#issuecomment-2315771804
@michaelritsema do you have intent to follow through with the questions and work here? If this is just a feature request, we'll move it to a GitHub Discussion or close it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org