You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@superset.apache.org by vi...@apache.org on 2022/04/08 13:47:57 UTC
[superset] branch 1.5 updated (031fadbed8 -> 77d6207bed)
This is an automated email from the ASF dual-hosted git repository.
villebro pushed a change to branch 1.5
in repository https://gitbox.apache.org/repos/asf/superset.git
from 031fadbed8 chore(releasing): use node 16 for testing and remove redundant updating note (#19505)
new 34e3119c40 fix(migrations): reorder skipped 1.4 migrations (#19543)
new c4d24a09a8 fix(sqllab): null database with backend persistence (#19548)
new d99a9a4134 fix: big number with trendline can't calculate cumsum (#19542)
new 1e567999d9 fix(dataset): avoid crash if database missing (#19582)
new 44eb81e35f fix(sqla): apply jinja to metrics (#19565)
new 5ca126698a fix(dataset): handle missing python_type gracefully (#19553)
new 280ecab0e6 chore: remove redundant adodbapi warning (#19557)
new ecaecf0b6c fix(select): render when empty multiselect (#19612)
new 77d6207bed docs: release notes for 1.5 (#19530)
The 9 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
RELEASING/README.md | 1 +
RELEASING/release-notes-1-5/README.md | 142 +++++++++++++++++++++
.../release-notes-1-5/media/adhoc_columns.png | Bin 0 -> 206451 bytes
.../release-notes-1-5/media/categorical_line.png | Bin 0 -> 308426 bytes
.../release-notes-1-5/media/dependent_filters.png | Bin 0 -> 437988 bytes
RELEASING/release-notes-1-5/media/filter_bar.png | Bin 0 -> 367513 bytes
RELEASING/release-notes-1-5/media/permalink.png | Bin 0 -> 187932 bytes
.../BigNumber/BigNumberWithTrendline/buildQuery.ts | 56 ++------
.../BigNumberWithTrendline/controlPanel.tsx | 46 +++++++
superset-frontend/src/SqlLab/actions/sqlLab.js | 21 ++-
.../src/components/Datasource/DatasourceEditor.jsx | 6 +-
superset-frontend/src/components/Select/Select.tsx | 17 ++-
superset-frontend/src/components/Select/utils.ts | 9 +-
.../nativeFilters/FilterBar/FilterBar.test.tsx | 3 +-
.../index.tsx | 14 +-
superset/connectors/sqla/models.py | 83 +++++++-----
superset/connectors/sqla/utils.py | 13 +-
superset/db_engine_specs/__init__.py | 3 +
...e27eaf93db_add_extra_config_column_to_alerts.py | 4 +-
.../versions/b8d3a24d9131_new_dataset_models.py | 14 +-
.../versions/b92d69a6643c_rename_csv_to_file.py | 4 +-
...7149153d_add_certifications_columns_to_slice.py | 4 +-
superset/models/sql_lab.py | 2 +-
23 files changed, 326 insertions(+), 116 deletions(-)
create mode 100644 RELEASING/release-notes-1-5/README.md
create mode 100644 RELEASING/release-notes-1-5/media/adhoc_columns.png
create mode 100644 RELEASING/release-notes-1-5/media/categorical_line.png
create mode 100644 RELEASING/release-notes-1-5/media/dependent_filters.png
create mode 100644 RELEASING/release-notes-1-5/media/filter_bar.png
create mode 100644 RELEASING/release-notes-1-5/media/permalink.png
[superset] 04/09: fix(dataset): avoid crash if database missing (#19582)
Posted by vi...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
villebro pushed a commit to branch 1.5
in repository https://gitbox.apache.org/repos/asf/superset.git
commit 1e567999d99d32b487697c3a81d50af206ce20dd
Author: Ville Brofeldt <33...@users.noreply.github.com>
AuthorDate: Thu Apr 7 13:43:17 2022 +0300
fix(dataset): avoid crash if database missing (#19582)
(cherry picked from commit db2135109a2b41240547653c845854422adaa92b)
---
superset-frontend/src/components/Datasource/DatasourceEditor.jsx | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/superset-frontend/src/components/Datasource/DatasourceEditor.jsx b/superset-frontend/src/components/Datasource/DatasourceEditor.jsx
index 1b6de1c7a9..15f9afa447 100644
--- a/superset-frontend/src/components/Datasource/DatasourceEditor.jsx
+++ b/superset-frontend/src/components/Datasource/DatasourceEditor.jsx
@@ -1001,10 +1001,10 @@ class DatasourceEditor extends React.PureComponent {
database={{
...datasource.database,
database_name:
- datasource.database.database_name ||
- datasource.database.name,
+ datasource.database?.database_name ||
+ datasource.database?.name,
}}
- dbId={datasource.database.id}
+ dbId={datasource.database?.id}
handleError={this.props.addDangerToast}
schema={datasource.schema}
sqlLabMode={false}
[superset] 09/09: docs: release notes for 1.5 (#19530)
Posted by vi...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
villebro pushed a commit to branch 1.5
in repository https://gitbox.apache.org/repos/asf/superset.git
commit 77d6207bed81e8bca7e3caaf342eacf10c09eff2
Author: Ville Brofeldt <33...@users.noreply.github.com>
AuthorDate: Thu Apr 7 12:52:20 2022 +0300
docs: release notes for 1.5 (#19530)
* docs: release notes for 1.5
* rephrase and remove secret key from 1.4
* address review comments
(cherry picked from commit 1c1beb653a52c1fcc67a97e539314f138117c6ba)
---
RELEASING/README.md | 1 +
RELEASING/release-notes-1-5/README.md | 142 +++++++++++++++++++++
.../release-notes-1-5/media/adhoc_columns.png | Bin 0 -> 206451 bytes
.../release-notes-1-5/media/categorical_line.png | Bin 0 -> 308426 bytes
.../release-notes-1-5/media/dependent_filters.png | Bin 0 -> 437988 bytes
RELEASING/release-notes-1-5/media/filter_bar.png | Bin 0 -> 367513 bytes
RELEASING/release-notes-1-5/media/permalink.png | Bin 0 -> 187932 bytes
7 files changed, 143 insertions(+)
diff --git a/RELEASING/README.md b/RELEASING/README.md
index 8724cd1642..32fb1aef34 100644
--- a/RELEASING/README.md
+++ b/RELEASING/README.md
@@ -30,6 +30,7 @@ partaking in the process should join the channel.
## Release notes for recent releases
+- [1.5](release-notes-1-5/README.md)
- [1.4](release-notes-1-4/README.md)
- [1.3](release-notes-1-3/README.md)
- [1.2](release-notes-1-2/README.md)
diff --git a/RELEASING/release-notes-1-5/README.md b/RELEASING/release-notes-1-5/README.md
new file mode 100644
index 0000000000..7444fa4d7c
--- /dev/null
+++ b/RELEASING/release-notes-1-5/README.md
@@ -0,0 +1,142 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# Release Notes for Superset 1.5
+
+Superset 1.5 focuses on polishing the dashboard native filters experience, while
+improving performance and stability. Superset 1.5 is likely the last minor release of
+version 1 of Superset, and will be succeeded by Superset 2.0. The 1.5 branch
+introduces the notion of a Long Term Support (LTS) version of Superset, and will
+receive security and other critical fixes even after Superset 2.x is released.
+Therefore, users will have the choice of staying on the 1.5 branch or upgrading to 2.x
+when available.
+
+- [**User Experience**](#user-facing-features)
+- [**Feature flags**](#feature-flags)
+- [**Database Experience**](#database-experience)
+- [**Breaking Changes and Full Changelog**](#breaking-changes-and-full-changelog)
+
+## User Facing Features
+
+- Complex dashboards with lots of native filters and charts will render considerably
+ faster. See the videos that shows the rendering time of a complex dashboard go from
+ 11 to 3 seconds: [#19064](https://github.com/apache/superset/pull/19064). In
+ addition, applying filters and switching tabs is also much smoother.
+- The Native Filter Bar has been redesigned, along with moving the "Apply" and
+ "Clear all" buttons to the bottom:
+
+
+
+- Native filters can now be made dependent on multiple filters. This makes it possible
+ to restrict the available values in a filter based on the selection of other filters.
+
+
+
+- In addition to being able to write Custom SQL for adhoc metrics and filters, the
+ column control now also features a Custom SQL tab. This makes it possible to write
+ custom expressions directly in charts without adding them to the dataset as saved
+ expressions.
+
+
+
+- A new `SupersetMetastoreCache` has been added which makes it possible to cache data
+ in the Superset Metastore without the need for running a dedicated cache like Redis
+ or Memcached. The new cache will be used by default for required caches, but can also
+ be used for caching chart or other data. See the
+ [documentation](https://superset.apache.org/docs/installation/cache#caching) for
+ details on using the new cache.
+- Previously it was possible for Dashboards with lots of filters to cause an error.
+ A similar issue existed on Explore. Now Superset stores Dashboard and Explore state
+ in the cache (as opposed to the URL), eliminating the infamous
+ [Long URL Problem](https://github.com/apache/superset/issues/17086).
+- Previously permanent links to Dashboard and Explore pages were in fact shortened URLS
+ that relied on state being stored in the URL (see Long URL Problem above). In
+ addition, the links used numerical ids and didn't check user permissions making it
+ easy to iterate through links that were stored in the metastore. Now permanent links
+ state is stored as JSON objects in the metastore, making it possible to store
+ arbitrarily large Dashboard and Explore state in permalinks. In addition, the ids
+ are encoded using [`hashids`](https://hashids.org/) and check permissions, making
+ permalink state more secure.
+
+
+
+## Feature flags
+
+- A new feature flag `GENERIC_CHART_AXES` has been added that makes it possible to
+ use a non-temporal x-axis on the ECharts Timeseries chart
+ ([#17917](https://github.com/apache/superset/pull/17917)). When enabled, a new
+ control "X Axis" is added to the control panel of ECharts line, area, bar, step and
+ scatter charts, which makes it possible to use categorical or numerical x-axes on
+ those charts.
+
+
+
+## Database Experience
+
+- DuckDB: Add support for database:
+ [#19317](https://github.com/apache/superset/pull/19317)
+
+- Kusto: Add support for Azure Data Explorer (Kusto):
+ [#17898](https://github.com/apache/superset/pull/17898)
+
+- Trino: Add server cert support and new auth methods:
+ [#17593](https://github.com/apache/superset/pull/17593) and
+ [#16346](https://github.com/apache/superset/pull/16346)
+
+- Microsoft SQL Server (MSSQL): support using CTEs in virtual tables:
+ [#18567](https://github.com/apache/superset/pull/18567)
+
+- Teradata and MSSQL: add support for TOP limit syntax:
+ [#18746](https://github.com/apache/superset/pull/18746) and
+ [#18240](https://github.com/apache/superset/pull/18240)
+
+- Apache Drill: User impersonation using `drill+sadrill`:
+ [#19252](https://github.com/apache/superset/pull/19252)
+
+## Developer Experience
+
+- `superset-ui` has now been integrated into the Superset codebase as per
+ [SIP-58](https://github.com/apache/superset/issues/13013) dubbed "Monorepo". This
+ makes development of plugins that ship with Superset considerably simpler. In
+ addition, it makes it possible to align `superset-ui` releases with official Superset
+ releases.
+
+## Breaking Changes and Full Changelog
+
+**Breaking Changes**
+
+- Bump `mysqlclient` from v1 to v2:
+ [#17556](https://github.com/apache/superset/pull/17556)
+- Single and double quotes will no longer be removed from filter values:
+ [#17881](https://github.com/apache/superset/pull/17881)
+- Previously `QUERY_COST_FORMATTERS_BY_ENGINE`, `SQL_VALIDATORS_BY_ENGINE` and
+ `SCHEDULED_QUERIES` were expected to be defined in the feature flag dictionary in
+ the `config.py` file. These should now be defined as a top-level config, with the
+ feature flag dictionary being reserved for boolean only values:
+ [#15254](https://github.com/apache/superset/pull/15254)
+- All Superset CLI commands (init, load_examples and etc) require setting the
+ `FLASK_APP` environment variable (which is set by default when `.flaskenv` is loaded):
+ [#17539](https://github.com/apache/superset/pull/17539)
+
+**Changelog**
+
+To see the complete changelog in this release, head to
+[CHANGELOG.MD](https://github.com/apache/superset/blob/1.5/CHANGELOG.md).
+As mentioned earlier, this release has a MASSIVE amount of bug fixes. The full
+changelog lists all of them!
diff --git a/RELEASING/release-notes-1-5/media/adhoc_columns.png b/RELEASING/release-notes-1-5/media/adhoc_columns.png
new file mode 100644
index 0000000000..6c73693625
Binary files /dev/null and b/RELEASING/release-notes-1-5/media/adhoc_columns.png differ
diff --git a/RELEASING/release-notes-1-5/media/categorical_line.png b/RELEASING/release-notes-1-5/media/categorical_line.png
new file mode 100644
index 0000000000..8d88aee15b
Binary files /dev/null and b/RELEASING/release-notes-1-5/media/categorical_line.png differ
diff --git a/RELEASING/release-notes-1-5/media/dependent_filters.png b/RELEASING/release-notes-1-5/media/dependent_filters.png
new file mode 100644
index 0000000000..a92afcf32c
Binary files /dev/null and b/RELEASING/release-notes-1-5/media/dependent_filters.png differ
diff --git a/RELEASING/release-notes-1-5/media/filter_bar.png b/RELEASING/release-notes-1-5/media/filter_bar.png
new file mode 100644
index 0000000000..61170fea6d
Binary files /dev/null and b/RELEASING/release-notes-1-5/media/filter_bar.png differ
diff --git a/RELEASING/release-notes-1-5/media/permalink.png b/RELEASING/release-notes-1-5/media/permalink.png
new file mode 100644
index 0000000000..8edebd4c45
Binary files /dev/null and b/RELEASING/release-notes-1-5/media/permalink.png differ
[superset] 02/09: fix(sqllab): null database with backend persistence (#19548)
Posted by vi...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
villebro pushed a commit to branch 1.5
in repository https://gitbox.apache.org/repos/asf/superset.git
commit c4d24a09a81c554c375d72d2746808c5a4a36132
Author: Ville Brofeldt <33...@users.noreply.github.com>
AuthorDate: Wed Apr 6 12:32:41 2022 +0300
fix(sqllab): null database with backend persistence (#19548)
(cherry picked from commit 2d81c4c79f93b9954d5090964b4f140bfb35723e)
---
superset-frontend/src/SqlLab/actions/sqlLab.js | 21 ++++++++++++++++-----
superset/models/sql_lab.py | 2 +-
2 files changed, 17 insertions(+), 6 deletions(-)
diff --git a/superset-frontend/src/SqlLab/actions/sqlLab.js b/superset-frontend/src/SqlLab/actions/sqlLab.js
index 02e07d5831..e602b796bd 100644
--- a/superset-frontend/src/SqlLab/actions/sqlLab.js
+++ b/superset-frontend/src/SqlLab/actions/sqlLab.js
@@ -1393,10 +1393,21 @@ export function queryEditorSetFunctionNames(queryEditor, dbId) {
functionNames: json.function_names,
}),
)
- .catch(() =>
- dispatch(
- addDangerToast(t('An error occurred while fetching function names.')),
- ),
- );
+ .catch(err => {
+ if (err.status === 404) {
+ // for databases that have been deleted, just reset the function names
+ dispatch({
+ type: QUERY_EDITOR_SET_FUNCTION_NAMES,
+ queryEditor,
+ functionNames: [],
+ });
+ } else {
+ dispatch(
+ addDangerToast(
+ t('An error occurred while fetching function names.'),
+ ),
+ );
+ }
+ });
};
}
diff --git a/superset/models/sql_lab.py b/superset/models/sql_lab.py
index 6a3b4ad8bf..57d7220705 100644
--- a/superset/models/sql_lab.py
+++ b/superset/models/sql_lab.py
@@ -118,7 +118,7 @@ class Query(Model, ExtraJSONMixin):
"changedOn": self.changed_on,
"changed_on": self.changed_on.isoformat(),
"dbId": self.database_id,
- "db": self.database.database_name,
+ "db": self.database.database_name if self.database else None,
"endDttm": self.end_time,
"errorMessage": self.error_message,
"executedSql": self.executed_sql,
[superset] 01/09: fix(migrations): reorder skipped 1.4 migrations (#19543)
Posted by vi...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
villebro pushed a commit to branch 1.5
in repository https://gitbox.apache.org/repos/asf/superset.git
commit 34e3119c40012dce19449c5279358578438a6e1d
Author: Ville Brofeldt <33...@users.noreply.github.com>
AuthorDate: Wed Apr 6 10:47:12 2022 +0300
fix(migrations): reorder skipped 1.4 migrations (#19543)
(cherry picked from commit e1ef2baad7bd2081e94af93b5487afe96a7b8292)
---
.../versions/abe27eaf93db_add_extra_config_column_to_alerts.py | 4 ++--
superset/migrations/versions/b92d69a6643c_rename_csv_to_file.py | 4 ++--
.../versions/f9847149153d_add_certifications_columns_to_slice.py | 4 ++--
3 files changed, 6 insertions(+), 6 deletions(-)
diff --git a/superset/migrations/versions/abe27eaf93db_add_extra_config_column_to_alerts.py b/superset/migrations/versions/abe27eaf93db_add_extra_config_column_to_alerts.py
index 5a20fc894a..2bc22cc2cf 100644
--- a/superset/migrations/versions/abe27eaf93db_add_extra_config_column_to_alerts.py
+++ b/superset/migrations/versions/abe27eaf93db_add_extra_config_column_to_alerts.py
@@ -17,14 +17,14 @@
"""add_extra_config_column_to_alerts
Revision ID: abe27eaf93db
-Revises: aea15018d53b
+Revises: 0ca9e5f1dacd
Create Date: 2021-12-02 12:03:20.691171
"""
# revision identifiers, used by Alembic.
revision = "abe27eaf93db"
-down_revision = "aea15018d53b"
+down_revision = "0ca9e5f1dacd"
import sqlalchemy as sa
from alembic import op
diff --git a/superset/migrations/versions/b92d69a6643c_rename_csv_to_file.py b/superset/migrations/versions/b92d69a6643c_rename_csv_to_file.py
index 1f94445ff4..b816b24320 100644
--- a/superset/migrations/versions/b92d69a6643c_rename_csv_to_file.py
+++ b/superset/migrations/versions/b92d69a6643c_rename_csv_to_file.py
@@ -17,14 +17,14 @@
"""rename_csv_to_file
Revision ID: b92d69a6643c
-Revises: 32646df09c64
+Revises: aea15018d53b
Create Date: 2021-09-19 14:42:20.130368
"""
# revision identifiers, used by Alembic.
revision = "b92d69a6643c"
-down_revision = "32646df09c64"
+down_revision = "aea15018d53b"
import sqlalchemy as sa
from alembic import op
diff --git a/superset/migrations/versions/f9847149153d_add_certifications_columns_to_slice.py b/superset/migrations/versions/f9847149153d_add_certifications_columns_to_slice.py
index a7e8a2239f..bc8f37fd03 100644
--- a/superset/migrations/versions/f9847149153d_add_certifications_columns_to_slice.py
+++ b/superset/migrations/versions/f9847149153d_add_certifications_columns_to_slice.py
@@ -17,7 +17,7 @@
"""add_certifications_columns_to_slice
Revision ID: f9847149153d
-Revises: 0ca9e5f1dacd
+Revises: 32646df09c64
Create Date: 2021-11-03 14:07:09.905194
"""
@@ -27,7 +27,7 @@ import sqlalchemy as sa
from alembic import op
revision = "f9847149153d"
-down_revision = "0ca9e5f1dacd"
+down_revision = "32646df09c64"
def upgrade():
[superset] 08/09: fix(select): render when empty multiselect (#19612)
Posted by vi...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
villebro pushed a commit to branch 1.5
in repository https://gitbox.apache.org/repos/asf/superset.git
commit ecaecf0b6c1fc0b33dedcfccd19b098a1c5264be
Author: Ville Brofeldt <33...@users.noreply.github.com>
AuthorDate: Fri Apr 8 16:40:14 2022 +0300
fix(select): render when empty multiselect (#19612)
* fix(select): render when empty multiselect
* disable flaky test
(cherry picked from commit 1ad82af058ec79a544f48df7a1aa9b0a165ecfb8)
---
superset-frontend/src/components/Select/Select.tsx | 17 ++++++++---------
superset-frontend/src/components/Select/utils.ts | 9 +++++++--
.../nativeFilters/FilterBar/FilterBar.test.tsx | 3 ++-
.../AdhocFilterEditPopoverSimpleTabContent/index.tsx | 14 +++++---------
4 files changed, 22 insertions(+), 21 deletions(-)
diff --git a/superset-frontend/src/components/Select/Select.tsx b/superset-frontend/src/components/Select/Select.tsx
index 9c7b92c38c..20bab6bcfc 100644
--- a/superset-frontend/src/components/Select/Select.tsx
+++ b/superset-frontend/src/components/Select/Select.tsx
@@ -42,7 +42,7 @@ import Icons from 'src/components/Icons';
import { getClientErrorObject } from 'src/utils/getClientErrorObject';
import { SLOW_DEBOUNCE } from 'src/constants';
import { rankedSearchCompare } from 'src/utils/rankedSearchCompare';
-import { getValue, hasOption, isObject } from './utils';
+import { getValue, hasOption, isLabeledValue } from './utils';
const { Option } = AntdSelect;
@@ -376,7 +376,7 @@ const Select = (
const missingValues: OptionsType = ensureIsArray(selectValue)
.filter(opt => !hasOption(getValue(opt), selectOptions))
.map(opt =>
- typeof opt === 'object' ? opt : { value: opt, label: String(opt) },
+ isLabeledValue(opt) ? opt : { value: opt, label: String(opt) },
);
return missingValues.length > 0
? missingValues.concat(selectOptions)
@@ -393,12 +393,11 @@ const Select = (
} else {
setSelectValue(previousState => {
const array = ensureIsArray(previousState);
- const isLabeledValue = isObject(selectedItem);
- const value = isLabeledValue ? selectedItem.value : selectedItem;
+ const value = getValue(selectedItem);
// Tokenized values can contain duplicated values
if (!hasOption(value, array)) {
const result = [...array, selectedItem];
- return isLabeledValue
+ return isLabeledValue(selectedItem)
? (result as AntdLabeledValue[])
: (result as (string | number)[]);
}
@@ -412,12 +411,12 @@ const Select = (
value: string | number | AntdLabeledValue | undefined,
) => {
if (Array.isArray(selectValue)) {
- if (typeof value === 'number' || typeof value === 'string' || !value) {
- const array = selectValue as (string | number)[];
- setSelectValue(array.filter(element => element !== value));
- } else {
+ if (isLabeledValue(value)) {
const array = selectValue as AntdLabeledValue[];
setSelectValue(array.filter(element => element.value !== value.value));
+ } else {
+ const array = selectValue as (string | number)[];
+ setSelectValue(array.filter(element => element !== value));
}
}
setInputValue('');
diff --git a/superset-frontend/src/components/Select/utils.ts b/superset-frontend/src/components/Select/utils.ts
index 73c6dd3533..9836b9ddd2 100644
--- a/superset-frontend/src/components/Select/utils.ts
+++ b/superset-frontend/src/components/Select/utils.ts
@@ -24,6 +24,7 @@ import {
OptionsType,
GroupedOptionsType,
} from 'react-select';
+import { LabeledValue as AntdLabeledValue } from 'antd/lib/select';
export function isObject(value: unknown): value is Record<string, unknown> {
return (
@@ -68,10 +69,14 @@ export function findValue<OptionType extends OptionTypeBase>(
return (Array.isArray(value) ? value : [value]).map(find);
}
+export function isLabeledValue(value: unknown): value is AntdLabeledValue {
+ return isObject(value) && 'value' in value && 'label' in value;
+}
+
export function getValue(
- option: string | number | { value: string | number | null } | null,
+ option: string | number | AntdLabeledValue | null | undefined,
) {
- return isObject(option) ? option.value : option;
+ return isLabeledValue(option) ? option.value : option;
}
type LabeledValue<V> = { label?: ReactNode; value?: V };
diff --git a/superset-frontend/src/dashboard/components/nativeFilters/FilterBar/FilterBar.test.tsx b/superset-frontend/src/dashboard/components/nativeFilters/FilterBar/FilterBar.test.tsx
index 632f8978ef..de7d6af99c 100644
--- a/superset-frontend/src/dashboard/components/nativeFilters/FilterBar/FilterBar.test.tsx
+++ b/superset-frontend/src/dashboard/components/nativeFilters/FilterBar/FilterBar.test.tsx
@@ -88,7 +88,8 @@ const addFilterFlow = async () => {
userEvent.click(screen.getByText('Time range'));
userEvent.type(screen.getByTestId(getModalTestId('name-input')), FILTER_NAME);
userEvent.click(screen.getByText('Save'));
- await screen.findByText('All filters (1)');
+ // TODO: fix this flaky test
+ // await screen.findByText('All filters (1)');
};
const addFilterSetFlow = async () => {
diff --git a/superset-frontend/src/explore/components/controls/FilterControl/AdhocFilterEditPopoverSimpleTabContent/index.tsx b/superset-frontend/src/explore/components/controls/FilterControl/AdhocFilterEditPopoverSimpleTabContent/index.tsx
index 4c521d8aad..58b1b25081 100644
--- a/superset-frontend/src/explore/components/controls/FilterControl/AdhocFilterEditPopoverSimpleTabContent/index.tsx
+++ b/superset-frontend/src/explore/components/controls/FilterControl/AdhocFilterEditPopoverSimpleTabContent/index.tsx
@@ -406,15 +406,11 @@ const AdhocFilterEditPopoverSimpleTabContent: React.FC<Props> = props => {
{...operatorSelectProps}
/>
{MULTI_OPERATORS.has(operatorId) || suggestions.length > 0 ? (
- // We need to delay rendering the select because we can't pass a primitive value without options
- // We can't pass value = [null] and options=[]
- comparatorSelectProps.value && suggestions.length === 0 ? null : (
- <SelectWithLabel
- labelText={labelText}
- options={suggestions}
- {...comparatorSelectProps}
- />
- )
+ <SelectWithLabel
+ labelText={labelText}
+ options={suggestions}
+ {...comparatorSelectProps}
+ />
) : (
<StyledInput
data-test="adhoc-filter-simple-value"
[superset] 03/09: fix: big number with trendline can't calculate cumsum (#19542)
Posted by vi...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
villebro pushed a commit to branch 1.5
in repository https://gitbox.apache.org/repos/asf/superset.git
commit d99a9a413401ed61f14797b9612e8c733f198b76
Author: Yongjie Zhao <yo...@gmail.com>
AuthorDate: Wed Apr 6 19:23:01 2022 +0800
fix: big number with trendline can't calculate cumsum (#19542)
(cherry picked from commit 2daa07163326b8555488dab523c5479cf92821cf)
---
.../BigNumber/BigNumberWithTrendline/buildQuery.ts | 56 +++++-----------------
.../BigNumberWithTrendline/controlPanel.tsx | 46 ++++++++++++++++++
2 files changed, 59 insertions(+), 43 deletions(-)
diff --git a/superset-frontend/plugins/plugin-chart-echarts/src/BigNumber/BigNumberWithTrendline/buildQuery.ts b/superset-frontend/plugins/plugin-chart-echarts/src/BigNumber/BigNumberWithTrendline/buildQuery.ts
index d55cf4664d..de75b50838 100644
--- a/superset-frontend/plugins/plugin-chart-echarts/src/BigNumber/BigNumberWithTrendline/buildQuery.ts
+++ b/superset-frontend/plugins/plugin-chart-echarts/src/BigNumber/BigNumberWithTrendline/buildQuery.ts
@@ -18,63 +18,33 @@
*/
import {
buildQueryContext,
- PostProcessingResample,
+ DTTM_ALIAS,
QueryFormData,
} from '@superset-ui/core';
import {
flattenOperator,
+ pivotOperator,
+ resampleOperator,
rollingWindowOperator,
- sortOperator,
} from '@superset-ui/chart-controls';
-const TIME_GRAIN_MAP: Record<string, string> = {
- PT1S: 'S',
- PT1M: 'min',
- PT5M: '5min',
- PT10M: '10min',
- PT15M: '15min',
- PT30M: '30min',
- PT1H: 'H',
- P1D: 'D',
- P1M: 'MS',
- P3M: 'QS',
- P1Y: 'AS',
- // TODO: these need to be mapped carefully, as the first day of week
- // can vary from engine to engine
- // P1W: 'W',
- // '1969-12-28T00:00:00Z/P1W': 'W',
- // '1969-12-29T00:00:00Z/P1W': 'W',
- // 'P1W/1970-01-03T00:00:00Z': 'W',
- // 'P1W/1970-01-04T00:00:00Z': 'W',
-};
-
export default function buildQuery(formData: QueryFormData) {
return buildQueryContext(formData, baseQueryObject => {
- // todo: move into full advanced analysis section here
- const rollingProc = rollingWindowOperator(formData, baseQueryObject);
- const { time_grain_sqla } = formData;
- let resampleProc: PostProcessingResample;
- if (rollingProc && time_grain_sqla) {
- const rule = TIME_GRAIN_MAP[time_grain_sqla];
- if (rule) {
- resampleProc = {
- operation: 'resample',
- options: {
- method: 'asfreq',
- rule,
- fill_value: null,
- },
- };
- }
- }
+ const { x_axis } = formData;
+ const is_timeseries = x_axis === DTTM_ALIAS || !x_axis;
+
return [
{
...baseQueryObject,
is_timeseries: true,
post_processing: [
- sortOperator(formData, baseQueryObject),
- resampleProc,
- rollingProc,
+ pivotOperator(formData, {
+ ...baseQueryObject,
+ index: x_axis,
+ is_timeseries,
+ }),
+ rollingWindowOperator(formData, baseQueryObject),
+ resampleOperator(formData, baseQueryObject),
flattenOperator(formData, baseQueryObject),
],
},
diff --git a/superset-frontend/plugins/plugin-chart-echarts/src/BigNumber/BigNumberWithTrendline/controlPanel.tsx b/superset-frontend/plugins/plugin-chart-echarts/src/BigNumber/BigNumberWithTrendline/controlPanel.tsx
index c1378543f6..6b99af91ce 100644
--- a/superset-frontend/plugins/plugin-chart-echarts/src/BigNumber/BigNumberWithTrendline/controlPanel.tsx
+++ b/superset-frontend/plugins/plugin-chart-echarts/src/BigNumber/BigNumberWithTrendline/controlPanel.tsx
@@ -217,6 +217,52 @@ const config: ControlPanelConfig = {
},
},
],
+ // eslint-disable-next-line react/jsx-key
+ [<h1 className="section-header">{t('Resample')}</h1>],
+ [
+ {
+ name: 'resample_rule',
+ config: {
+ type: 'SelectControl',
+ freeForm: true,
+ label: t('Rule'),
+ default: null,
+ choices: [
+ ['1T', '1 minutely frequency'],
+ ['1H', '1 hourly frequency'],
+ ['1D', '1 calendar day frequency'],
+ ['7D', '7 calendar day frequency'],
+ ['1MS', '1 month start frequency'],
+ ['1M', '1 month end frequency'],
+ ['1AS', '1 year start frequency'],
+ ['1A', '1 year end frequency'],
+ ],
+ description: t('Pandas resample rule'),
+ },
+ },
+ ],
+ [
+ {
+ name: 'resample_method',
+ config: {
+ type: 'SelectControl',
+ freeForm: true,
+ label: t('Fill method'),
+ default: null,
+ choices: [
+ ['asfreq', 'Null imputation'],
+ ['zerofill', 'Zero imputation'],
+ ['linear', 'Linear interpolation'],
+ ['ffill', 'Forward values'],
+ ['bfill', 'Backward values'],
+ ['median', 'Median values'],
+ ['mean', 'Mean values'],
+ ['sum', 'Sum values'],
+ ],
+ description: t('Pandas resample method'),
+ },
+ },
+ ],
],
},
],
[superset] 05/09: fix(sqla): apply jinja to metrics (#19565)
Posted by vi...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
villebro pushed a commit to branch 1.5
in repository https://gitbox.apache.org/repos/asf/superset.git
commit 44eb81e35f769b2f3f3224a3c8e6b2045e48e5cc
Author: Ville Brofeldt <33...@users.noreply.github.com>
AuthorDate: Thu Apr 7 14:04:51 2022 +0300
fix(sqla): apply jinja to metrics (#19565)
(cherry picked from commit 34b55765c4b0cbd8f0b9f89c6ca0f62f4478270e)
---
superset/connectors/sqla/models.py | 83 ++++++++++++++++++++++++--------------
1 file changed, 52 insertions(+), 31 deletions(-)
diff --git a/superset/connectors/sqla/models.py b/superset/connectors/sqla/models.py
index 8721f6ea81..b8d3a7d091 100644
--- a/superset/connectors/sqla/models.py
+++ b/superset/connectors/sqla/models.py
@@ -354,6 +354,7 @@ class TableColumn(Model, BaseColumn, CertificationMixin):
:param time_grain: Optional time grain, e.g. P1Y
:param label: alias/label that column is expected to have
+ :param template_processor: template processor
:return: A TimeExpression object wrapped in a Label if supported by db
"""
label = label or utils.DTTM_ALIAS
@@ -517,6 +518,27 @@ sqlatable_user = Table(
)
+def _process_sql_expression(
+ expression: Optional[str],
+ database_id: int,
+ schema: str,
+ template_processor: Optional[BaseTemplateProcessor],
+) -> Optional[str]:
+ if template_processor and expression:
+ expression = template_processor.process_template(expression)
+ if expression:
+ expression = validate_adhoc_subquery(
+ expression,
+ database_id,
+ schema,
+ )
+ try:
+ expression = sanitize_clause(expression)
+ except QueryClauseValidationException as ex:
+ raise QueryObjectValidationError(ex.message) from ex
+ return expression
+
+
class SqlaTable(Model, BaseDatasource): # pylint: disable=too-many-public-methods
"""An ORM object for SqlAlchemy table references"""
@@ -899,13 +921,17 @@ class SqlaTable(Model, BaseDatasource): # pylint: disable=too-many-public-metho
return sql
def adhoc_metric_to_sqla(
- self, metric: AdhocMetric, columns_by_name: Dict[str, TableColumn]
+ self,
+ metric: AdhocMetric,
+ columns_by_name: Dict[str, TableColumn],
+ template_processor: Optional[BaseTemplateProcessor] = None,
) -> ColumnElement:
"""
Turn an adhoc metric into a sqlalchemy column.
:param dict metric: Adhoc metric definition
:param dict columns_by_name: Columns for the current table
+ :param template_processor: template_processor instance
:returns: The metric defined as a sqlalchemy column
:rtype: sqlalchemy.sql.column
"""
@@ -922,17 +948,12 @@ class SqlaTable(Model, BaseDatasource): # pylint: disable=too-many-public-metho
sqla_column = column(column_name)
sqla_metric = self.sqla_aggregations[metric["aggregate"]](sqla_column)
elif expression_type == utils.AdhocMetricExpressionType.SQL:
- tp = self.get_template_processor()
- expression = tp.process_template(cast(str, metric["sqlExpression"]))
- expression = validate_adhoc_subquery(
- expression,
- self.database_id,
- self.schema,
+ expression = _process_sql_expression(
+ expression=metric["sqlExpression"],
+ database_id=self.database_id,
+ schema=self.schema,
+ template_processor=template_processor,
)
- try:
- expression = sanitize_clause(expression)
- except QueryClauseValidationException as ex:
- raise QueryObjectValidationError(ex.message) from ex
sqla_metric = literal_column(expression)
else:
raise QueryObjectValidationError("Adhoc metric expressionType is invalid")
@@ -953,21 +974,14 @@ class SqlaTable(Model, BaseDatasource): # pylint: disable=too-many-public-metho
:rtype: sqlalchemy.sql.column
"""
label = utils.get_column_name(col)
- expression = col["sqlExpression"]
- if template_processor and expression:
- expression = template_processor.process_template(expression)
- if expression:
- expression = validate_adhoc_subquery(
- expression,
- self.database_id,
- self.schema,
- )
- try:
- expression = sanitize_clause(expression)
- except QueryClauseValidationException as ex:
- raise QueryObjectValidationError(ex.message) from ex
- sqla_metric = literal_column(expression)
- return self.make_sqla_column_compatible(sqla_metric, label)
+ expression = _process_sql_expression(
+ expression=col["sqlExpression"],
+ database_id=self.database_id,
+ schema=self.schema,
+ template_processor=template_processor,
+ )
+ sqla_column = literal_column(expression)
+ return self.make_sqla_column_compatible(sqla_column, label)
def make_sqla_column_compatible(
self, sqla_col: ColumnElement, label: Optional[str] = None
@@ -1151,7 +1165,13 @@ class SqlaTable(Model, BaseDatasource): # pylint: disable=too-many-public-metho
for metric in metrics:
if utils.is_adhoc_metric(metric):
assert isinstance(metric, dict)
- metrics_exprs.append(self.adhoc_metric_to_sqla(metric, columns_by_name))
+ metrics_exprs.append(
+ self.adhoc_metric_to_sqla(
+ metric=metric,
+ columns_by_name=columns_by_name,
+ template_processor=template_processor,
+ )
+ )
elif isinstance(metric, str) and metric in metrics_by_name:
metrics_exprs.append(metrics_by_name[metric].get_sqla_col())
else:
@@ -1178,10 +1198,11 @@ class SqlaTable(Model, BaseDatasource): # pylint: disable=too-many-public-metho
if isinstance(col, dict):
col = cast(AdhocMetric, col)
if col.get("sqlExpression"):
- col["sqlExpression"] = validate_adhoc_subquery(
- cast(str, col["sqlExpression"]),
- self.database_id,
- self.schema,
+ col["sqlExpression"] = _process_sql_expression(
+ expression=col["sqlExpression"],
+ database_id=self.database_id,
+ schema=self.schema,
+ template_processor=template_processor,
)
if utils.is_adhoc_metric(col):
# add adhoc sort by column to columns_by_name if not exists
[superset] 06/09: fix(dataset): handle missing python_type gracefully (#19553)
Posted by vi...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
villebro pushed a commit to branch 1.5
in repository https://gitbox.apache.org/repos/asf/superset.git
commit 5ca126698a941965f291e0243fa67d65686e4ac5
Author: Ville Brofeldt <33...@users.noreply.github.com>
AuthorDate: Thu Apr 7 14:30:11 2022 +0300
fix(dataset): handle missing python_type gracefully (#19553)
* fix(dataset): handle missing python_type gracefully
* refactor TEMPORAL_TYPES
(cherry picked from commit d9343a463980cf8b09ed394554fb54200027cc70)
---
superset/connectors/sqla/utils.py | 13 ++++++++++---
.../migrations/versions/b8d3a24d9131_new_dataset_models.py | 14 +++++++++++---
2 files changed, 21 insertions(+), 6 deletions(-)
diff --git a/superset/connectors/sqla/utils.py b/superset/connectors/sqla/utils.py
index 14b9071d1d..a2b54201d6 100644
--- a/superset/connectors/sqla/utils.py
+++ b/superset/connectors/sqla/utils.py
@@ -15,6 +15,7 @@
# specific language governing permissions and limitations
# under the License.
from contextlib import closing
+from datetime import date, datetime, time, timedelta
from typing import Callable, Dict, List, Optional, Set, TYPE_CHECKING
import sqlparse
@@ -41,7 +42,7 @@ if TYPE_CHECKING:
from superset.connectors.sqla.models import SqlaTable
-TEMPORAL_TYPES = {"DATETIME", "DATE", "TIME", "TIMEDELTA"}
+TEMPORAL_TYPES = {date, datetime, time, timedelta}
def get_physical_table_metadata(
@@ -172,6 +173,13 @@ def validate_adhoc_subquery(
return ";\n".join(str(statement) for statement in statements)
+def is_column_type_temporal(column_type: TypeEngine) -> bool:
+ try:
+ return column_type.python_type in TEMPORAL_TYPES
+ except NotImplementedError:
+ return False
+
+
def load_or_create_tables( # pylint: disable=too-many-arguments
session: Session,
database_id: int,
@@ -223,8 +231,7 @@ def load_or_create_tables( # pylint: disable=too-many-arguments
name=column["name"],
type=str(column["type"]),
expression=conditional_quote(column["name"]),
- is_temporal=column["type"].python_type.__name__.upper()
- in TEMPORAL_TYPES,
+ is_temporal=is_column_type_temporal(column["type"]),
is_aggregation=False,
is_physical=True,
is_spatial=False,
diff --git a/superset/migrations/versions/b8d3a24d9131_new_dataset_models.py b/superset/migrations/versions/b8d3a24d9131_new_dataset_models.py
index 75f5293034..22aa2f451c 100644
--- a/superset/migrations/versions/b8d3a24d9131_new_dataset_models.py
+++ b/superset/migrations/versions/b8d3a24d9131_new_dataset_models.py
@@ -25,6 +25,7 @@ Create Date: 2021-11-11 16:41:53.266965
"""
import json
+from datetime import date, datetime, time, timedelta
from typing import Callable, List, Optional, Set
from uuid import uuid4
@@ -35,6 +36,7 @@ from sqlalchemy.engine.url import make_url
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import backref, relationship, Session
from sqlalchemy.schema import UniqueConstraint
+from sqlalchemy.sql.type_api import TypeEngine
from sqlalchemy_utils import UUIDType
from superset import app, db
@@ -230,7 +232,14 @@ class NewDataset(Base):
external_url = sa.Column(sa.Text, nullable=True)
-TEMPORAL_TYPES = {"DATETIME", "DATE", "TIME", "TIMEDELTA"}
+TEMPORAL_TYPES = {date, datetime, time, timedelta}
+
+
+def is_column_type_temporal(column_type: TypeEngine) -> bool:
+ try:
+ return column_type.python_type in TEMPORAL_TYPES
+ except NotImplementedError:
+ return False
def load_or_create_tables(
@@ -285,8 +294,7 @@ def load_or_create_tables(
name=column["name"],
type=str(column["type"]),
expression=conditional_quote(column["name"]),
- is_temporal=column["type"].python_type.__name__.upper()
- in TEMPORAL_TYPES,
+ is_temporal=is_column_type_temporal(column["type"]),
is_aggregation=False,
is_physical=True,
is_spatial=False,
[superset] 07/09: chore: remove redundant adodbapi warning (#19557)
Posted by vi...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
villebro pushed a commit to branch 1.5
in repository https://gitbox.apache.org/repos/asf/superset.git
commit 280ecab0e6a7a7828fe2425b67cc68ba6795f3d3
Author: Ville Brofeldt <33...@users.noreply.github.com>
AuthorDate: Thu Apr 7 14:30:36 2022 +0300
chore: remove redundant adodbapi warning (#19557)
(cherry picked from commit 0d331f5bd81f25bc92a20da6ed8d99f0c393efb2)
---
superset/db_engine_specs/__init__.py | 3 +++
1 file changed, 3 insertions(+)
diff --git a/superset/db_engine_specs/__init__.py b/superset/db_engine_specs/__init__.py
index 9eeb87acfe..4474f2a748 100644
--- a/superset/db_engine_specs/__init__.py
+++ b/superset/db_engine_specs/__init__.py
@@ -116,6 +116,9 @@ def get_available_engine_specs() -> Dict[Type[BaseEngineSpec], Set[str]]:
hasattr(attribute, "dialect")
and inspect.isclass(attribute.dialect)
and issubclass(attribute.dialect, DefaultDialect)
+ # adodbapi dialect is removed in SQLA 1.4 and doesn't implement the
+ # `dbapi` method, hence needs to be ignored to avoid logging a warning
+ and attribute.dialect.driver != "adodbapi"
):
try:
attribute.dialect.dbapi()