You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@superset.apache.org by be...@apache.org on 2023/08/11 15:21:43 UTC
[superset] branch fix_dos updated (ad89b839f5 -> 57f29842aa)
This is an automated email from the ASF dual-hosted git repository.
beto pushed a change to branch fix_dos
in repository https://gitbox.apache.org/repos/asf/superset.git
omit ad89b839f5 Add some comments and change values
omit 546230402a Address comments
omit 51e7e1bdc9 chore: rate limit requests
add 764f0f79ca docs: Fixing Superset typo in docker-compose local installation guide (#24920)
add 284c12697b chore: Removes duplicated featureFlags.ts (#24935)
add 94c595093b chore: Add explicit ON DELETE CASCADE for dashboard_slices (#24938)
add 55014bf58b chore: Add explicit ON DELETE CASCADE for embedded_dashboards (#24939)
add bcd24936bc fix: remove unused file (#24946)
add ce65a3b9cd fix(charts): View in SQL Lab with relevant perm (#24903)
add 41ca4a00b9 fix: `to_datetime` in Pandas 2 (#24952)
add aca006f38b fix: timezone issue in Pandas 2 (#24955)
add 6f24a4e7a8 fix: calls to `_get_sqla_engine` (#24953)
add f5ed4072e4 feat(gsheets): file upload (#24921)
new 50dabf3994 chore: rate limit requests
new 0d1bdb0791 Address comments
new e9bba52cb5 Add some comments and change values
new 57f29842aa Disable on tests
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (ad89b839f5)
\
N -- N -- N refs/heads/fix_dos (57f29842aa)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 4 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
UPDATING.md | 2 +
_update-notifier-last-checked | 0
.../installing-superset-using-docker-compose.mdx | 4 +-
requirements/development.txt | 2 +-
requirements/testing.in | 2 +-
requirements/testing.txt | 7 +-
.../superset-ui-core/src/utils/featureFlags.ts | 10 +-
.../test/utils/featureFlag.test.ts | 52 +++++--
superset-frontend/src/SqlLab/App.jsx | 8 +-
superset-frontend/src/SqlLab/actions/sqlLab.js | 8 +-
.../src/SqlLab/actions/sqlLab.test.js | 6 +-
.../ShareSqlLabQuery/ShareSqlLabQuery.test.tsx | 6 +-
.../SqlLab/components/ShareSqlLabQuery/index.tsx | 9 +-
.../src/SqlLab/components/SouthPane/index.tsx | 3 +-
.../src/SqlLab/components/SqlEditor/index.jsx | 2 +-
.../SqlLab/components/TabbedSqlEditors/index.jsx | 3 +-
.../components/TableElement/TableElement.test.tsx | 4 +-
superset-frontend/src/components/Chart/Chart.jsx | 2 +-
.../src/components/Chart/chartAction.js | 9 +-
.../src/components/Datasource/DatasourceEditor.jsx | 2 +-
.../Datasource/DatasourceEditor.test.jsx | 6 +-
.../components/Datasource/DatasourceModal.test.jsx | 5 +-
.../src/components/Datasource/DatasourceModal.tsx | 3 +-
.../src/components/DynamicPlugins/index.tsx | 2 +-
.../components/ReportModal/ReportModal.test.tsx | 6 +-
.../src/dashboard/actions/dashboardState.js | 2 +-
.../src/dashboard/actions/dashboardState.test.js | 4 +-
superset-frontend/src/dashboard/actions/hydrate.js | 3 +-
.../src/dashboard/actions/sliceEntities.ts | 2 +-
.../DashboardBuilder/DashboardBuilder.test.tsx | 42 +++---
.../DashboardBuilder/DashboardBuilder.tsx | 2 +-
.../dashboard/components/DashboardBuilder/state.ts | 3 +-
.../HeaderActionsDropdown.test.tsx | 6 +-
.../Header/HeaderActionsDropdown/index.jsx | 10 +-
.../src/dashboard/components/Header/index.jsx | 2 +-
.../PropertiesModal/PropertiesModal.test.tsx | 3 +-
.../dashboard/components/PropertiesModal/index.tsx | 2 +-
.../components/SliceHeaderControls/index.tsx | 2 +-
.../FilterBar/FilterControls/FilterValue.tsx | 2 +-
.../nativeFilters/FilterBar/Vertical.tsx | 2 +-
.../FiltersConfigForm/FiltersConfigForm.tsx | 2 +-
.../components/nativeFilters/utils.test.ts | 8 +-
.../dashboard/components/nativeFilters/utils.ts | 2 +-
.../src/dashboard/util/permissionUtils.test.ts | 71 ++++++---
.../src/dashboard/util/permissionUtils.ts | 20 ++-
superset-frontend/src/dataMask/actions.ts | 2 +-
superset-frontend/src/dataMask/reducer.ts | 2 +-
.../components/DataTablesPane/DataTablesPane.tsx | 9 +-
.../explore/components/DatasourcePanel/index.tsx | 2 +-
.../explore/components/ExploreChartPanel/index.jsx | 2 +-
.../src/explore/components/SaveModal.tsx | 2 +-
.../DatasourceControl/DatasourceControl.test.tsx | 2 +-
.../controls/DatasourceControl/index.jsx | 4 +-
...AdhocFilterEditPopoverSimpleTabContent.test.tsx | 4 +-
.../index.tsx | 2 +-
.../useExploreAdditionalActionsMenu/index.jsx | 10 +-
superset-frontend/src/featureFlags.ts | 37 -----
.../src/features/alerts/AlertReportModal.tsx | 2 +-
.../src/features/charts/ChartCard.tsx | 3 +-
.../src/features/dashboards/DashboardCard.tsx | 3 +-
superset-frontend/src/features/tags/TagCard.tsx | 3 +-
.../src/middleware/asyncEvent.test.ts | 4 +-
superset-frontend/src/middleware/asyncEvent.ts | 2 +-
.../src/pages/ChartCreation/index.tsx | 2 +-
.../src/pages/ChartList/ChartList.test.jsx | 6 +-
superset-frontend/src/pages/ChartList/index.tsx | 2 +-
.../src/pages/DashboardList/DashboardList.test.jsx | 6 +-
.../src/pages/DashboardList/index.tsx | 9 +-
superset-frontend/src/pages/DatabaseList/index.tsx | 2 +-
.../src/pages/DatasetList/DatasetList.test.tsx | 4 +-
superset-frontend/src/pages/DatasetList/index.tsx | 2 +-
superset-frontend/src/pages/Home/Home.test.tsx | 6 +-
superset-frontend/src/pages/Home/index.tsx | 10 +-
.../pages/SavedQueryList/SavedQueryList.test.jsx | 4 +-
.../src/pages/SavedQueryList/index.tsx | 9 +-
superset-frontend/src/pages/Tags/index.tsx | 3 +-
superset-frontend/src/preamble.ts | 8 +-
superset-frontend/src/utils/hostNamesConfig.js | 2 +-
superset-frontend/src/views/routes.test.tsx | 4 -
superset/daos/chart.py | 3 -
superset/daos/dashboard.py | 6 -
superset/db_engine_specs/gsheets.py | 162 +++++++++++++++++++--
superset/db_engine_specs/trino.py | 7 +-
..._add_on_delete_cascade_for_dashboard_slices.py} | 37 ++---
...n_delete_cascade_for_embedded_dashboards.py.py} | 36 ++---
superset/models/core.py | 51 +++----
superset/models/dashboard.py | 4 +-
superset/models/embedded_dashboard.py | 6 +-
superset/result_set.py | 9 +-
superset/utils/core.py | 10 +-
tests/integration_tests/celery_tests.py | 5 +-
tests/integration_tests/charts/api_tests.py | 1 -
tests/integration_tests/charts/data/api_tests.py | 7 +-
tests/integration_tests/dashboard_tests.py | 3 -
tests/integration_tests/databases/api_tests.py | 2 +-
tests/integration_tests/model_tests.py | 3 +-
.../security/guest_token_security_tests.py | 2 -
tests/integration_tests/superset_test_config.py | 2 +
tests/integration_tests/tagging_tests.py | 2 -
tests/integration_tests/utils_tests.py | 6 +-
tests/unit_tests/databases/api_test.py | 4 +-
tests/unit_tests/db_engine_specs/test_gsheets.py | 92 ++++++++++++
tests/unit_tests/result_set_test.py | 26 +++-
tests/unit_tests/utils/test_core.py | 30 ++++
104 files changed, 672 insertions(+), 371 deletions(-)
delete mode 100644 _update-notifier-last-checked
delete mode 100644 superset-frontend/src/featureFlags.ts
copy superset/migrations/versions/{2023-06-22_13-39_6fbe660cac39_add_on_delete_cascade_for_tables_references.py => 2023-08-09_14-17_8ace289026f3_add_on_delete_cascade_for_dashboard_slices.py} (64%)
copy superset/migrations/versions/{2023-06-22_13-39_6fbe660cac39_add_on_delete_cascade_for_tables_references.py => 2023-08-09_15-39_4448fa6deeb1__dd_on_delete_cascade_for_embedded_dashboards.py.py} (61%)
[superset] 03/04: Add some comments and change values
Posted by be...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
beto pushed a commit to branch fix_dos
in repository https://gitbox.apache.org/repos/asf/superset.git
commit e9bba52cb5f7d0a8816138c97774bdafcd86886b
Author: Beto Dealmeida <ro...@dealmeida.net>
AuthorDate: Wed Aug 9 18:02:26 2023 -0700
Add some comments and change values
---
superset/config.py | 14 +++++++++++---
1 file changed, 11 insertions(+), 3 deletions(-)
diff --git a/superset/config.py b/superset/config.py
index 18cbccfd05..0b70328e0b 100644
--- a/superset/config.py
+++ b/superset/config.py
@@ -266,11 +266,19 @@ PROXY_FIX_CONFIG = {"x_for": 1, "x_proto": 1, "x_host": 1, "x_port": 1, "x_prefi
# Configuration for scheduling queries from SQL Lab.
SCHEDULED_QUERIES: dict[str, Any] = {}
-# FAB Rate limiting
+# FAB Rate limiting: this is a security feature for preventing DDOS attacks. The
+# feature is on by default to make Superset secure by default, but you should
+# fine tune the limits to your needs. You can read more about the different
+# parameters here: https://flask-limiter.readthedocs.io/en/stable/configuration.html
RATELIMIT_ENABLED = True
-AUTH_RATE_LIMITED = True
-AUTH_RATE_LIMIT = "2 per 5 second"
RATELIMIT_APPLICATION = "50 per second"
+AUTH_RATE_LIMITED = True
+AUTH_RATE_LIMIT = "5 per second"
+# A storage location conforming to the scheme in storage-scheme. See the limits
+# library for allowed values: https://limits.readthedocs.io/en/stable/storage.html
+# RATELIMIT_STORAGE_URI = "redis://host:port"
+# A callable that returns the unique identity of the current request.
+# RATELIMIT_REQUEST_IDENTIFIER = flask.Request.endpoint
# ------------------------------
# GLOBALS FOR APP Builder
[superset] 02/04: Address comments
Posted by be...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
beto pushed a commit to branch fix_dos
in repository https://gitbox.apache.org/repos/asf/superset.git
commit 0d1bdb07918f49f36c09eddd71a3aab0060a59ea
Author: Beto Dealmeida <ro...@dealmeida.net>
AuthorDate: Thu Jul 6 17:47:52 2023 -0700
Address comments
---
superset/config.py | 2 +-
superset/dashboards/api.py | 2 +-
superset/models/dashboard.py | 5 +++--
superset/utils/dashboard_import_export.py | 4 ++--
superset/views/dashboard/views.py | 2 +-
5 files changed, 8 insertions(+), 7 deletions(-)
diff --git a/superset/config.py b/superset/config.py
index d430273008..18cbccfd05 100644
--- a/superset/config.py
+++ b/superset/config.py
@@ -266,7 +266,7 @@ PROXY_FIX_CONFIG = {"x_for": 1, "x_proto": 1, "x_host": 1, "x_port": 1, "x_prefi
# Configuration for scheduling queries from SQL Lab.
SCHEDULED_QUERIES: dict[str, Any] = {}
-# Rate limiting
+# FAB Rate limiting
RATELIMIT_ENABLED = True
AUTH_RATE_LIMITED = True
AUTH_RATE_LIMIT = "2 per 5 second"
diff --git a/superset/dashboards/api.py b/superset/dashboards/api.py
index 1602c8e2f9..b2aa43b0ee 100644
--- a/superset/dashboards/api.py
+++ b/superset/dashboards/api.py
@@ -817,7 +817,7 @@ class DashboardRestApi(BaseSupersetModelRestApi):
Dashboard.id.in_(requested_ids)
)
query = self._base_filters.apply_all(query)
- ids = [item.id for item in query.all()]
+ ids = {item.id for item in query.all()}
if not ids:
return self.response_404()
export = Dashboard.export_dashboards(ids)
diff --git a/superset/models/dashboard.py b/superset/models/dashboard.py
index 5159fa873b..f837c76610 100644
--- a/superset/models/dashboard.py
+++ b/superset/models/dashboard.py
@@ -373,11 +373,12 @@ class Dashboard(Model, AuditMixinNullable, ImportExportMixin):
@classmethod
def export_dashboards( # pylint: disable=too-many-locals
- cls, dashboard_ids: list[int]
+ cls,
+ dashboard_ids: set[int],
) -> str:
copied_dashboards = []
datasource_ids = set()
- for dashboard_id in set(dashboard_ids):
+ for dashboard_id in dashboard_ids:
# make sure that dashboard_id is an integer
dashboard_id = int(dashboard_id)
dashboard = (
diff --git a/superset/utils/dashboard_import_export.py b/superset/utils/dashboard_import_export.py
index fc61d0a422..eef8cbe6df 100644
--- a/superset/utils/dashboard_import_export.py
+++ b/superset/utils/dashboard_import_export.py
@@ -27,8 +27,8 @@ def export_dashboards(session: Session) -> str:
"""Returns all dashboards metadata as a json dump"""
logger.info("Starting export")
dashboards = session.query(Dashboard)
- dashboard_ids = []
+ dashboard_ids = set()
for dashboard in dashboards:
- dashboard_ids.append(dashboard.id)
+ dashboard_ids.add(dashboard.id)
data = Dashboard.export_dashboards(dashboard_ids)
return data
diff --git a/superset/views/dashboard/views.py b/superset/views/dashboard/views.py
index a96d56fc14..ce5e8f1e07 100644
--- a/superset/views/dashboard/views.py
+++ b/superset/views/dashboard/views.py
@@ -78,7 +78,7 @@ class DashboardModelView(
@expose("/export_dashboards_form")
def download_dashboards(self) -> FlaskResponse:
if request.args.get("action") == "go":
- ids = request.args.getlist("id")
+ ids = set(request.args.getlist("id"))
return Response(
DashboardModel.export_dashboards(ids),
headers=generate_download_headers("json"),
[superset] 01/04: chore: rate limit requests
Posted by be...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
beto pushed a commit to branch fix_dos
in repository https://gitbox.apache.org/repos/asf/superset.git
commit 50dabf39941402a72cae7c50a15ab4b4eaee4433
Author: Beto Dealmeida <ro...@dealmeida.net>
AuthorDate: Thu Jun 1 15:57:36 2023 -0700
chore: rate limit requests
---
superset/config.py | 6 ++++++
superset/models/dashboard.py | 2 +-
2 files changed, 7 insertions(+), 1 deletion(-)
diff --git a/superset/config.py b/superset/config.py
index 75fda6eb37..d430273008 100644
--- a/superset/config.py
+++ b/superset/config.py
@@ -266,6 +266,12 @@ PROXY_FIX_CONFIG = {"x_for": 1, "x_proto": 1, "x_host": 1, "x_port": 1, "x_prefi
# Configuration for scheduling queries from SQL Lab.
SCHEDULED_QUERIES: dict[str, Any] = {}
+# Rate limiting
+RATELIMIT_ENABLED = True
+AUTH_RATE_LIMITED = True
+AUTH_RATE_LIMIT = "2 per 5 second"
+RATELIMIT_APPLICATION = "50 per second"
+
# ------------------------------
# GLOBALS FOR APP Builder
# ------------------------------
diff --git a/superset/models/dashboard.py b/superset/models/dashboard.py
index 719a6df8e4..5159fa873b 100644
--- a/superset/models/dashboard.py
+++ b/superset/models/dashboard.py
@@ -377,7 +377,7 @@ class Dashboard(Model, AuditMixinNullable, ImportExportMixin):
) -> str:
copied_dashboards = []
datasource_ids = set()
- for dashboard_id in dashboard_ids:
+ for dashboard_id in set(dashboard_ids):
# make sure that dashboard_id is an integer
dashboard_id = int(dashboard_id)
dashboard = (
[superset] 04/04: Disable on tests
Posted by be...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
beto pushed a commit to branch fix_dos
in repository https://gitbox.apache.org/repos/asf/superset.git
commit 57f29842aaceb00533acaf104a790617f766ca25
Author: Beto Dealmeida <ro...@dealmeida.net>
AuthorDate: Fri Aug 11 08:17:22 2023 -0700
Disable on tests
---
tests/integration_tests/superset_test_config.py | 2 ++
1 file changed, 2 insertions(+)
diff --git a/tests/integration_tests/superset_test_config.py b/tests/integration_tests/superset_test_config.py
index 77e007a2dd..bcc3146083 100644
--- a/tests/integration_tests/superset_test_config.py
+++ b/tests/integration_tests/superset_test_config.py
@@ -97,6 +97,8 @@ REDIS_CELERY_DB = os.environ.get("REDIS_CELERY_DB", 2)
REDIS_RESULTS_DB = os.environ.get("REDIS_RESULTS_DB", 3)
REDIS_CACHE_DB = os.environ.get("REDIS_CACHE_DB", 4)
+RATELIMIT_ENABLED = False
+
CACHE_CONFIG = {
"CACHE_TYPE": "RedisCache",