You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "shrimad-mishra-cognoai (via GitHub)" <gi...@apache.org> on 2025/05/14 17:34:24 UTC
[D] Issue with Incremental Dataset Refresh and Auto Time Grain [superset]
GitHub user shrimad-mishra-cognoai created a discussion: Issue with Incremental Dataset Refresh and Auto Time Grain
### Bug description
I made a virtual dataset with custom SQL queries, now I wanted to set the Time Grain to Auto in Chat or Dashboard and along with this I want to set the incremental refresh for the dataset.
Can any guide me how to do this?
### Screenshots/recordings
_No response_
### Superset version
master / latest-dev
### Python version
3.9
### Node version
16
### Browser
Chrome
### Additional context
_No response_
### Checklist
- [ ] I have searched Superset docs and Slack and didn't find a solution to my problem.
- [ ] I have searched the GitHub issue tracker and didn't find a similar bug report.
- [ ] I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.
GitHub link: https://github.com/apache/superset/discussions/33439
----
This is an automatically sent email for notifications@superset.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@superset.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [D] Issue with Incremental Dataset Refresh and Auto Time Grain [superset]
Posted by "dosubot[bot] (via GitHub)" <gi...@apache.org>.
GitHub user dosubot[bot] added a comment to the discussion: Issue with Incremental Dataset Refresh and Auto Time Grain
<!-- Answer -->
To set the Time Grain to Auto in Superset, you can configure it through the `superset/config.py` file. This involves using the `TIME_GRAIN_DENYLIST`, `TIME_GRAIN_ADDONS`, and `TIME_GRAIN_ADDON_EXPRESSIONS` configurations to manage time grains. The frontend component `TimeGrainFilterPlugin.tsx` handles the user interface for selecting time grains, allowing users to apply them dynamically in the UI <sup>[[1]](https://github.com/apache/superset/blob/master/superset/config.py)</sup><sup>[[2]](https://github.com/apache/superset/blob/master/superset-frontend/src/filters/components/TimeGrain/TimeGrainFilterPlugin.tsx)</sup>.
Unfortunately, I couldn't find specific information on how Superset handles incremental dataset refreshes or how to configure them. It might be necessary to explore the Superset documentation or other resources for guidance on this feature. If you have any specific questions about the code components or configurations involved, feel free to ask!
<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*
---
<sup>Help Dosu learn! Give it feedback: [Great Response](https://app.dosu.dev/response-feedback/73754f48-23a2-4f68-9145-931a05428278?feedback_type=great_response) | [Irrelevant Answer](https://app.dosu.dev/response-feedback/73754f48-23a2-4f68-9145-931a05428278?feedback_type=irrelevant_answer) | [Incorrect Sources](https://app.dosu.dev/response-feedback/73754f48-23a2-4f68-9145-931a05428278?feedback_type=incorrect_sources) | [Too Verbose](https://app.dosu.dev/response-feedback/73754f48-23a2-4f68-9145-931a05428278?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/73754f48-23a2-4f68-9145-931a05428278?feedback_type=hallucination) | [Bug Report](https://app.dosu.dev/response-feedback/73754f48-23a2-4f68-9145-931a05428278?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/73754f48-23a2-4f68-9145-931a05428278?feedback_type=other)</sup>
GitHub link: https://github.com/apache/superset/discussions/33439#discussioncomment-13148217
----
This is an automatically sent email for notifications@superset.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@superset.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [D] Issue with Incremental Dataset Refresh and Auto Time Grain [superset]
Posted by "dosubot[bot] (via GitHub)" <gi...@apache.org>.
GitHub user dosubot[bot] edited a comment on the discussion: Issue with Incremental Dataset Refresh and Auto Time Grain
<!-- Answer -->
I couldn't find specific information on known limitations or issues regarding the 'Auto' time grain feature in Superset within the provided context. It's possible that the details might be available elsewhere or could have been missed. You might want to explore the Superset documentation or codebase further for more insights on this feature.
<!-- Dosu Comment Footer -->
*To reply, just mention [@dosu](https://go.dosu.dev/dosubot?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=reply-with-mention&utm_term=apache%2Fsuperset).*
---
Share context across your team and agents. Try [Dosu](https://dosu.dev?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=knowledge-infrastructure-tagline&utm_term=apache%2Fsuperset).
[](https://app.dosu.dev/response-feedback?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=knowledge-infrastructure-feedback&utm_term=apache%2Fsuperset&message_id=c73043b0-f2cd-47ca-9a19-ca8aa18e9318) [](https://github.dosu.com/apache/superset?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=knowledge-infrastructure-learn-repo&utm_term=apache%2Fsuperset) [](https://app.dosu.dev/signup?referrer=openSource&source=github-footer&utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=knowledge-infrastructure-add-team&utm_term=apache%2Fsuperset)
GitHub link: https://github.com/apache/superset/discussions/33439#discussioncomment-13153323
----
This is an automatically sent email for notifications@superset.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@superset.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [D] Issue with Incremental Dataset Refresh and Auto Time Grain [superset]
Posted by "dosubot[bot] (via GitHub)" <gi...@apache.org>.
GitHub user dosubot[bot] edited a comment on the discussion: Issue with Incremental Dataset Refresh and Auto Time Grain
<!-- Answer -->
I couldn't find specific information on the exact configuration needed in `config.py` for the 'Auto' time grain feature in Superset. Additionally, details on how Superset handles incremental dataset refreshes and their configurations are not available in the provided context.
However, for managing virtual datasets with custom SQL queries, it's important to define time grain expressions specific to your database's SQL dialect. You can add new time grains via configuration by specifying them in `TIME_GRAIN_ADDONS` and implementing them in `TIME_GRAIN_ADDON_EXPRESSIONS` <sup>[[1]](https://github.com/apache/superset/blob/master/superset/db_engine_specs/README.md)</sup><sup>[[2]](https://github.com/apache/superset/blob/master/superset/db_engine_specs/sqlite.py)</sup>.
If you need further assistance, you might want to explore the Superset documentation or the codebase for more detailed guidance on these features.
<!-- Dosu Comment Footer -->
*To reply, just mention [@dosu](https://go.dosu.dev/dosubot?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=reply-with-mention&utm_term=apache%2Fsuperset).*
---
Docs are dead. Just use [Dosu](https://dosu.dev?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=static-docs-tagline&utm_term=apache%2Fsuperset).
[](https://app.dosu.dev/response-feedback?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=static-docs-feedback&utm_term=apache%2Fsuperset&message_id=89c0b2f6-fd81-458d-a4fc-7fc620be6a2b) [](https://github.dosu.com/apache/superset?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=static-docs-ask-repo&utm_term=apache%2Fsuperset) [](https://app.dosu.dev/signup?referrer=openSource&source=github-footer&utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=static-docs-share-team&utm_term=apache%2Fsuperset)
GitHub link: https://github.com/apache/superset/discussions/33439#discussioncomment-13148241
----
This is an automatically sent email for notifications@superset.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@superset.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [D] Issue with Incremental Dataset Refresh and Auto Time Grain [superset]
Posted by "dosubot[bot] (via GitHub)" <gi...@apache.org>.
GitHub user dosubot[bot] added a comment to the discussion: Issue with Incremental Dataset Refresh and Auto Time Grain
<!-- Answer -->
To enable Auto time grain in Superset, you need to configure the `TIME_GRAIN_ADDONS` and `TIME_GRAIN_ADDON_EXPRESSIONS` in the `config.py` file. These configurations allow you to define custom time grains and their corresponding SQL expressions for different database engines. Here's an example of how you might configure these:
```python
# superset_config.py
TIME_GRAIN_ADDONS = {
"PT2S": "2 second", # Example of adding a custom time grain
}
TIME_GRAIN_ADDON_EXPRESSIONS = {
"clickhouse": {
"PT2S": "toDateTime(intDiv(toUInt32(toDateTime({col})), 2)*2)", # Custom expression for ClickHouse
}
}
```
These configurations allow Superset to recognize and utilize custom time grains, which can be used in the Auto time grain feature. The default time grain setting that affects the Auto time grain feature is `'P1D'`, which stands for a period of 1 day <sup>[[1]](https://github.com/apache/superset/blob/master/superset/db_engine_specs/README.md)</sup><sup>[[2]](https://github.com/apache/superset/blob/master/superset-frontend/src/explore/controls.jsx)</sup>.
<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*
---
<sup>Help Dosu learn! Give it feedback: [Great Response](https://app.dosu.dev/response-feedback/c94515ec-e74a-4d4b-823b-8badc979a5e2?feedback_type=great_response) | [Irrelevant Answer](https://app.dosu.dev/response-feedback/c94515ec-e74a-4d4b-823b-8badc979a5e2?feedback_type=irrelevant_answer) | [Incorrect Sources](https://app.dosu.dev/response-feedback/c94515ec-e74a-4d4b-823b-8badc979a5e2?feedback_type=incorrect_sources) | [Too Verbose](https://app.dosu.dev/response-feedback/c94515ec-e74a-4d4b-823b-8badc979a5e2?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/c94515ec-e74a-4d4b-823b-8badc979a5e2?feedback_type=hallucination) | [Bug Report](https://app.dosu.dev/response-feedback/c94515ec-e74a-4d4b-823b-8badc979a5e2?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/c94515ec-e74a-4d4b-823b-8badc979a5e2?feedback_type=other)</sup>
GitHub link: https://github.com/apache/superset/discussions/33439#discussioncomment-13148219
----
This is an automatically sent email for notifications@superset.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@superset.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [D] Issue with Incremental Dataset Refresh and Auto Time Grain [superset]
Posted by "shrimad-mishra-cognoai (via GitHub)" <gi...@apache.org>.
GitHub user shrimad-mishra-cognoai added a comment to the discussion: Issue with Incremental Dataset Refresh and Auto Time Grain
@dosu can tell me exact config I need to have in the config.py for Auto time grain
GitHub link: https://github.com/apache/superset/discussions/33439#discussioncomment-13148218
----
This is an automatically sent email for notifications@superset.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@superset.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [D] Issue with Incremental Dataset Refresh and Auto Time Grain [superset]
Posted by "shrimad-mishra-cognoai (via GitHub)" <gi...@apache.org>.
GitHub user shrimad-mishra-cognoai added a comment to the discussion: Issue with Incremental Dataset Refresh and Auto Time Grain
@dosu Is this the known limitation in Superset where we cannot auto time grain feature?
GitHub link: https://github.com/apache/superset/discussions/33439#discussioncomment-13153317
----
This is an automatically sent email for notifications@superset.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@superset.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [D] Issue with Incremental Dataset Refresh and Auto Time Grain [superset]
Posted by "dosubot[bot] (via GitHub)" <gi...@apache.org>.
GitHub user dosubot[bot] added a comment to the discussion: Issue with Incremental Dataset Refresh and Auto Time Grain
<!-- Answer -->
I couldn't find specific information on known limitations or issues regarding the 'Auto' time grain feature in Superset within the provided context. It's possible that the details might be available elsewhere or could have been missed. You might want to explore the Superset documentation or codebase further for more insights on this feature.
<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*
---
<sup>Help Dosu learn! Give it feedback: [Great Response](https://app.dosu.dev/response-feedback/c73043b0-f2cd-47ca-9a19-ca8aa18e9318?feedback_type=great_response) | [Irrelevant Answer](https://app.dosu.dev/response-feedback/c73043b0-f2cd-47ca-9a19-ca8aa18e9318?feedback_type=irrelevant_answer) | [Incorrect Sources](https://app.dosu.dev/response-feedback/c73043b0-f2cd-47ca-9a19-ca8aa18e9318?feedback_type=incorrect_sources) | [Too Verbose](https://app.dosu.dev/response-feedback/c73043b0-f2cd-47ca-9a19-ca8aa18e9318?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/c73043b0-f2cd-47ca-9a19-ca8aa18e9318?feedback_type=hallucination) | [Bug Report](https://app.dosu.dev/response-feedback/c73043b0-f2cd-47ca-9a19-ca8aa18e9318?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/c73043b0-f2cd-47ca-9a19-ca8aa18e9318?feedback_type=other)</sup>
GitHub link: https://github.com/apache/superset/discussions/33439#discussioncomment-13153323
----
This is an automatically sent email for notifications@superset.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@superset.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [D] Issue with Incremental Dataset Refresh and Auto Time Grain [superset]
Posted by "dosubot[bot] (via GitHub)" <gi...@apache.org>.
GitHub user dosubot[bot] added a comment to the discussion: Issue with Incremental Dataset Refresh and Auto Time Grain
<!-- Answer -->
I couldn't find specific information on the exact configuration needed in `config.py` for the 'Auto' time grain feature in Superset. Additionally, details on how Superset handles incremental dataset refreshes and their configurations are not available in the provided context.
However, for managing virtual datasets with custom SQL queries, it's important to define time grain expressions specific to your database's SQL dialect. You can add new time grains via configuration by specifying them in `TIME_GRAIN_ADDONS` and implementing them in `TIME_GRAIN_ADDON_EXPRESSIONS` <sup>[[1]](https://github.com/apache/superset/blob/master/superset/db_engine_specs/README.md)</sup><sup>[[2]](https://github.com/apache/superset/blob/master/superset/db_engine_specs/sqlite.py)</sup>.
If you need further assistance, you might want to explore the Superset documentation or the codebase for more detailed guidance on these features.
<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*
---
<sup>Help Dosu learn! Give it feedback: [Great Response](https://app.dosu.dev/response-feedback/89c0b2f6-fd81-458d-a4fc-7fc620be6a2b?feedback_type=great_response) | [Irrelevant Answer](https://app.dosu.dev/response-feedback/89c0b2f6-fd81-458d-a4fc-7fc620be6a2b?feedback_type=irrelevant_answer) | [Incorrect Sources](https://app.dosu.dev/response-feedback/89c0b2f6-fd81-458d-a4fc-7fc620be6a2b?feedback_type=incorrect_sources) | [Too Verbose](https://app.dosu.dev/response-feedback/89c0b2f6-fd81-458d-a4fc-7fc620be6a2b?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/89c0b2f6-fd81-458d-a4fc-7fc620be6a2b?feedback_type=hallucination) | [Bug Report](https://app.dosu.dev/response-feedback/89c0b2f6-fd81-458d-a4fc-7fc620be6a2b?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/89c0b2f6-fd81-458d-a4fc-7fc620be6a2b?feedback_type=other)</sup>
GitHub link: https://github.com/apache/superset/discussions/33439#discussioncomment-13148241
----
This is an automatically sent email for notifications@superset.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@superset.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org