You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@superset.apache.org by be...@apache.org on 2020/12/08 01:07:46 UTC
[incubator-superset] branch PISQL-2 updated (b934e6e -> 990ca8a)
This is an automated email from the ASF dual-hosted git repository.
beto pushed a change to branch PISQL-2
in repository https://gitbox.apache.org/repos/asf/incubator-superset.git.
omit b934e6e Use new hook
omit dc6fc0f WIP
omit f34a3d2 Use new hook
omit 2be6e6f Fix test
omit fd978bb Fix lint
omit fd73fb7 Fix old importer
omit dad3b89 Improve text
omit 0c6c01a feat: add modal to import datasets
omit 90569fd Remove log and needless store.
omit 9eada98 Improve hook
omit 4e7a189 Fix test
omit 91f3b14 feat: add modal to import databases
add 13c51d5 chore: Fix TableModelView super call (#11815)
add bac84a3 fix: delete chart, dashboards, dbs with assoc reports (#11801)
add 1591d75 fix(setup): disallow python 3.6 for pip install (#11825)
add 3578410 docs(release): small fixes and deprecate apache docs (#11827)
add f121107 ci(docker): build image on release (#11828)
add 43c69d5 fix: Adding and removing annotations (#11811)
add ec01691 fix: wrong key used FEATURE_FLAGS[LISTVIEW_DEFAULT_CARD_VIEW] (#11824)
add 671ff6c fix: improve Avatar text size consistency (#11837)
add e974b0b Change subtabs order for Data tab (#11834)
add 19e5ce0 chore: Move styles of Dashboard and FilterScopeSelector modals to Emotion (#11779)
add 4666445 chore: type a couple easy to type files (#11838)
add 9dd33d5 feat(saved queries): security perm simplification (#11764)
add 5ebc09b fix: Download as image not working on Dashboard view (#11778)
add 0689738 feat: Sqllab to Explore UX improvements api changes (#11836)
add 16e69f1 feat: Option to run npm dev-server (hot reload) while calling docker-compose up (#11238)
add 462b39d fix: update chart doesn't remove all connections to dashboards (#11830)
add b1a963b fix: security converge downgrade procedure (#11852)
add a4f4425 Fix chart select borders in BuilderComponentPane (#11766)
add 0e9898c feat(welcome): add SQL snippets to saved queries card (#11678)
add a0b8135 fix(datasets): renaming openAPI columns (#11858)
add 3cd94d6 feat: Create api abstraction layer for dataset network calls (#11855)
add 0117247 feat: list view filters for Query History (#11702)
add 84e0606 chore: move fixtures and helpers to root (#11812)
add 44e80e0 chore: Add home link to navbar (#11851)
add ffa32cd chore: remove SIP_34_QUERY_SEARCH_UI feature flag (#11863)
add 3035090 fix: filters push out apply button on dashboard (#11580)
add a5af2ad test: add React Testing Library (#11771)
add ac9761c fix: dashboard iframe to markdown db migration (#11845)
add f292015 fix (dataset editor): allow Source tab readOnly mode (#11781)
add d041d3a fix: certification icon order (#11881)
add a76eadd feat: report/alert list CRUD view (#11802)
add 9514be5 style: remove react bootstrap fade component (#11843)
add 618a73d Align icon (#11848)
add 0728878 fix: Show copy icon (#11847)
add f2d9f00 feat: allow user to view their profile again (#11816)
add 1842c56 Capture error when celery back-end does not support forgetting tasks (#10770)
add 0c10f9e chore: Update sqllab types (#11887)
add 721f911 chore: Enable jsx-a11y/anchor-is-valid (#11891)
add 61064b9 add fix for extra comments for #11781 (#11902)
add 71c567b feat(css templates): security perm simplification (#11856)
add 8c063ef fix(query): sorting by user for Query History list view (#11897)
add 38b720b style: eslint curly rule (#11913)
add 53017e5 fix: Button storybook component size knob (#11912)
add 38d21ac fix: Check if annotation type is supported by the given chart type (#11876)
add 0396c70 chore: Support SET & SHOW commands as read only SQL commands (#11868)
add 9ae2f95 feat: Adding SaveDatasetModal component (#11861)
add 04f993e fix: adjust width for non multi select inputs (#11911)
add 54bf707 feat: run BQ queries as single statement (#11904)
add 5b19398 refactor: Move ProgressBar to Antd (#11875)
add e0288bf chore: add unit tests for the current import functionality (#11786)
add 41d4fb9 fix: Forgot to remove bad comments from SaveDatasetModal (#11919)
add 327a281 feat: add event and interval annotation support to chart data ep (#11665)
add 77d362d fix: Remove expensive logs table migration (#11920)
add 60122a2 feat: filters for alerts and reports list view (#11900)
add 98dc657 Added check if SQL query is empty and disable shortcut for run query (#11877)
add 8adc8b8 Quick fix for branch names in tags (#11930)
add fa68288 fix: make sure we log the logs (#11939)
add 64d6fb9 refactor: Extract copy functionality to utils (#11928)
add 2c32342 refactor: Move Badge to Antdesign (#11899)
add 66cd565 feat: add Postgres SQL validator (#11538)
add 41738df feat: Results table on Explore view (#11854)
add f98ae01 feat(releasing): support changelog csv export (#11893)
add 14ea444 fix(charts): screenshot API declaring an unnecessary rison arg (#11931)
add a7bba92 chore: bump nvd3 plugin version (#11947)
add 2b9695c feat: add modal to import databases (#11884)
add fbb458f feat: add modal to import datasets (#11910)
new 990ca8a WIP
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (b934e6e)
\
N -- N -- N refs/heads/PISQL-2 (990ca8a)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.github/workflows/bashlib.sh | 2 +
.github/workflows/docker.yml | 9 +-
.github/workflows/docker_build_push.sh | 7 +-
Dockerfile | 1 +
RELEASING/README.md | 42 +-
RELEASING/changelog.py | 106 +-
RELEASING/make_docs.sh | 55 -
RELEASING/make_docs_entrypoint.sh | 26 -
UPDATING.md | 3 +-
docker-compose.yml | 2 +-
.../docker-frontend.sh | 12 +-
requirements/base.txt | 37 +-
requirements/development.txt | 10 +-
requirements/docker.txt | 2 +-
requirements/integration.txt | 12 +-
requirements/testing.txt | 8 +-
setup.cfg | 2 +-
setup.py | 12 +-
superset-frontend/.eslintrc.js | 10 +-
.../explore/visualizations/big_number.test.js | 6 +-
superset-frontend/images/empty-query.svg | 25 +
.../images/icons/{grid.svg => exclamation.svg} | 4 +-
superset-frontend/images/icons/slack.svg | 27 +
superset-frontend/jest.config.js | 5 +-
superset-frontend/package-lock.json | 5057 ++++++++++++++------
superset-frontend/package.json | 16 +-
superset-frontend/spec/.eslintrc | 9 +-
.../dashboard => }/fixtures/mockChartQueries.js | 2 +-
.../dashboard => }/fixtures/mockDashboardData.js | 0
.../fixtures/mockDashboardFilters.js | 0
.../dashboard => }/fixtures/mockDashboardInfo.js | 0
.../dashboard => }/fixtures/mockDashboardLayout.js | 0
.../dashboard => }/fixtures/mockDashboardState.js | 0
.../dashboard => }/fixtures/mockSliceEntities.js | 2 +-
.../dashboard => }/fixtures/mockState.js | 4 +-
.../dashboard => }/fixtures/mockStore.js | 0
.../dashboard => }/helpers/WithDragDropContext.jsx | 0
.../columnType.js => spec/helpers/setup.ts} | 12 +-
superset-frontend/spec/helpers/shim.ts | 13 +
.../helpers/testing-library.tsx} | 39 +-
.../spec/javascripts/CRUD/CollectionTable_spec.jsx | 2 +-
.../components/ListView/ListView_spec.jsx | 64 +-
.../spec/javascripts/components/Menu_spec.jsx | 11 +
.../spec/javascripts/components/Timer_spec.tsx | 85 +
.../dashboard/actions/dashboardState_spec.js | 6 +-
.../dashboard/components/DashboardBuilder_spec.jsx | 6 +-
.../dashboard/components/Dashboard_spec.jsx | 12 +-
.../dashboard/components/FiltersBadge_spec.tsx | 8 +-
.../dashboard/components/PropertiesModal_spec.jsx | 2 +-
.../dashboard/components/SliceAdder_spec.jsx | 2 +-
.../components/gridComponents/ChartHolder_spec.jsx | 8 +-
.../components/gridComponents/Chart_spec.jsx | 6 +-
.../components/gridComponents/Column_spec.jsx | 6 +-
.../components/gridComponents/Divider_spec.jsx | 2 +-
.../components/gridComponents/Header_spec.jsx | 4 +-
.../components/gridComponents/Markdown_spec.jsx | 6 +-
.../components/gridComponents/Row_spec.jsx | 6 +-
.../components/gridComponents/Tab_spec.jsx | 6 +-
.../components/gridComponents/Tabs_spec.jsx | 6 +-
.../new/DraggableNewComponent_spec.jsx | 2 +-
.../dashboard/reducers/dashboardFilters_spec.js | 6 +-
.../util/getLeafComponentIdFromPath_spec.js | 4 +-
.../util/updateComponentParentsList_spec.js | 2 +-
.../datasource/ChangeDatasourceModal_spec.jsx | 2 +-
.../datasource/DatasourceEditor_spec.jsx | 66 +-
.../datasource/DatasourceModal_spec.jsx | 2 +-
.../{ControlRow_spec.jsx => ControlRow_spec.tsx} | 18 +-
.../explore/components/DisplayQueryButton_spec.jsx | 4 +-
.../components/FilterBoxItemControl_spec.jsx | 2 +-
.../spec/javascripts/sqllab/ResultSet_spec.jsx | 3 +-
.../javascripts/sqllab/SaveDatasetModal_spec.tsx | 51 +
.../spec/javascripts/sqllab/TableElement_spec.jsx | 9 +
.../spec/javascripts/sqllab/Timer_spec.jsx | 48 -
.../views/CRUD/alert/AlertList_spec.jsx | 111 +
.../CRUD/csstemplates/CssTemplatesList_spec.jsx | 4 +-
.../CRUD/data/savedquery/SavedQueryList_spec.jsx | 2 +-
superset-frontend/src/SqlLab/components/App.jsx | 5 +-
.../src/SqlLab/components/QueryTable.jsx | 9 +-
.../src/SqlLab/components/ResultSet.tsx | 6 +-
.../src/SqlLab/components/SaveDatasetModal.tsx | 180 +
.../src/SqlLab/components/SqlEditor.jsx | 12 +-
.../src/SqlLab/components/TableElement.jsx | 5 +-
.../src/SqlLab/components/TemplateParamsEditor.jsx | 4 +-
superset-frontend/src/SqlLab/types.ts | 4 +
superset-frontend/src/api/dataset.ts | 61 +
superset-frontend/src/chart/Chart.jsx | 2 +
superset-frontend/src/chart/chartAction.js | 5 +-
.../Loading.tsx => common/components/Badge.tsx} | 43 +-
.../common/components/{Popover.tsx => Fade.tsx} | 11 +-
.../src/common/components/ProgressBar.tsx | 54 +
.../components/Switch.tsx} | 16 +-
.../src/common/components/common.stories.tsx | 24 +
superset-frontend/src/common/components/index.tsx | 10 +-
.../src/components/Button/Button.stories.jsx | 2 +-
.../src/components/CopyToClipboard.jsx | 63 +-
.../src/components/DatabaseSelector.tsx | 7 +-
.../src/components/FacePile/index.tsx | 23 +-
superset-frontend/src/components/FaveStar.tsx | 2 +
superset-frontend/src/components/Icon/index.tsx | 62 +-
superset-frontend/src/components/Label/index.tsx | 1 +
.../ListView/Filters/Base.ts} | 23 +-
.../src/components/ListView/Filters/DateRange.tsx | 75 +
.../src/components/ListView/Filters/Search.tsx | 61 +
.../ListView/{Filters.tsx => Filters/Select.tsx} | 140 +-
.../src/components/ListView/Filters/index.tsx | 108 +
superset-frontend/src/components/ListView/types.ts | 69 +-
superset-frontend/src/components/ListView/utils.ts | 62 +-
.../src/components/ListViewCard/index.tsx | 4 +-
superset-frontend/src/components/Menu/Menu.tsx | 14 +-
superset-frontend/src/components/ModalTrigger.jsx | 1 +
.../src/components/Select/OnPasteSelect.jsx | 8 +-
superset-frontend/src/components/Select/styles.tsx | 11 +-
superset-frontend/src/components/TableSelector.tsx | 36 +-
superset-frontend/src/components/Timer.tsx | 10 +-
.../dashboard/components/BuilderComponentPane.jsx | 12 +-
.../dashboard/components/DeleteComponentModal.jsx | 80 -
.../dashboard/components/HeaderActionsDropdown.jsx | 26 +-
.../dashboard/components/RefreshIntervalModal.jsx | 10 +-
.../dashboard/components/SliceHeaderControls.jsx | 20 +-
.../components/filterscope/FilterScopeModal.jsx | 12 +-
.../components/filterscope/FilterScopeSelector.jsx | 24 +-
.../dashboard/components/gridComponents/Chart.jsx | 2 +-
.../dashboard/stylesheets/components/chart.less | 5 +
.../src/dashboard/stylesheets/dashboard.less | 36 -
.../stylesheets/filter-scope-selector.less | 7 -
.../src/datasource/DatasourceEditor.jsx | 87 +-
.../src/explore/components/DataTableControl.tsx | 105 +
.../src/explore/components/DataTablesPane.tsx | 245 +
.../src/explore/components/DisplayQueryButton.jsx | 146 +-
.../src/explore/components/ExploreChartPanel.jsx | 253 +-
.../components/controls/AnnotationLayer.jsx | 102 +-
.../components/controls/AnnotationLayerControl.jsx | 53 +-
.../components/controls/TextAreaControl.jsx | 6 +-
superset-frontend/src/featureFlags.ts | 2 +-
.../src/profile/components/Security.tsx | 11 +-
.../src/setup/{setupClient.js => setupClient.ts} | 8 +-
.../{setupFormatters.js => setupFormatters.ts} | 0
.../{setupPluginsExtra.js => setupPluginsExtra.ts} | 0
superset-frontend/src/utils/copy.ts | 56 +
superset-frontend/src/utils/downloadAsImage.ts | 3 +-
.../utils/{errorMessages.js => errorMessages.ts} | 0
superset-frontend/src/views/App.tsx | 23 +-
.../src/views/CRUD/alert/AlertList.tsx | 365 ++
.../src/views/CRUD/{annotation => alert}/types.ts | 26 +-
.../src/views/CRUD/annotation/AnnotationModal.tsx | 1 -
.../views/CRUD/csstemplates/CssTemplatesList.tsx | 6 +-
superset-frontend/src/views/CRUD/data/common.ts | 15 +-
.../components/SyntaxHighlighterCopy/index.tsx | 41 +-
.../src/views/CRUD/data/database/DatabaseList.tsx | 8 +-
.../src/views/CRUD/data/dataset/DatasetList.tsx | 14 +-
.../src/views/CRUD/data/query/QueryList.test.tsx | 29 +-
.../src/views/CRUD/data/query/QueryList.tsx | 140 +-
.../CRUD/data/query/QueryPreviewModal.test.tsx | 12 +-
.../views/CRUD/data/savedquery/SavedQueryList.tsx | 45 +-
superset-frontend/src/views/CRUD/hooks.ts | 52 +-
superset-frontend/src/views/CRUD/types.ts | 20 +
superset-frontend/src/views/CRUD/utils.tsx | 17 +-
.../src/views/CRUD/welcome/SavedQueries.tsx | 123 +-
superset-frontend/webpack.proxy-config.js | 1 +
superset/app.py | 15 +-
superset/charts/api.py | 1 -
superset/charts/commands/bulk_delete.py | 10 +
superset/charts/commands/delete.py | 10 +
superset/charts/commands/exceptions.py | 10 +-
superset/charts/commands/update.py | 13 +-
superset/charts/schemas.py | 5 +-
superset/cli.py | 4 +-
superset/common/query_context.py | 100 +-
superset/common/query_object.py | 30 +-
superset/config.py | 12 +-
superset/connectors/sqla/models.py | 8 +-
superset/connectors/sqla/views.py | 4 +-
superset/constants.py | 33 +
superset/css_templates/api.py | 6 +-
superset/dashboards/commands/bulk_delete.py | 10 +
superset/dashboards/commands/delete.py | 10 +
superset/dashboards/commands/exceptions.py | 8 +
superset/databases/commands/delete.py | 11 +
superset/databases/commands/exceptions.py | 4 +
superset/datasets/api.py | 14 +-
superset/datasets/commands/importers/v0.py | 33 +-
superset/datasets/commands/update.py | 31 +-
superset/datasets/dao.py | 15 +-
superset/db_engine_specs/base.py | 12 +-
superset/db_engine_specs/bigquery.py | 4 +
superset/db_engine_specs/hive.py | 12 +-
superset/db_engine_specs/presto.py | 7 +-
.../migrations/shared}/__init__.py | 0
superset/migrations/shared/security_converge.py | 273 ++
.../utils.py} | 40 +-
...6_.py => 811494c0cc23_remove_path_from_logs.py} | 26 +-
...8ee129739cf9_security_converge_css_templates.py | 81 +
...978245563a02_migrate_iframe_to_dash_markdown.py | 17 +-
.../versions/a8173232b786_add_path_to_logs.py | 20 +-
...e38177dbf641_security_converge_saved_queries.py | 85 +
superset/models/core.py | 3 -
superset/queries/api.py | 10 +-
superset/queries/saved_queries/api.py | 6 +-
superset/reports/api.py | 5 +-
superset/reports/dao.py | 48 +
superset/reports/schemas.py | 2 +-
superset/sql_lab.py | 16 +-
superset/sql_parse.py | 20 +-
superset/sql_validators/__init__.py | 7 +-
superset/sql_validators/postgres.py | 54 +
superset/templates/appbuilder/navbar_right.html | 3 +-
superset/utils/core.py | 7 +
superset/utils/log.py | 19 +-
superset/views/alerts.py | 33 +
superset/views/base.py | 4 +
superset/views/core.py | 16 +-
superset/views/css_templates.py | 8 +-
superset/views/sql_lab.py | 8 +-
superset/viz.py | 17 +-
tests/annotation_layers/api_tests.py | 79 +-
tests/annotation_layers/fixtures.py | 101 +
tests/charts/api_tests.py | 182 +-
tests/core_tests.py | 5 +-
tests/css_templates/api_tests.py | 14 +
tests/dashboards/api_tests.py | 71 +
tests/dashboards/commands_tests.py | 57 +-
tests/databases/api_tests.py | 45 +
tests/datasets/api_tests.py | 38 +
tests/datasets/commands_tests.py | 98 +-
tests/db_engine_specs/base_engine_spec_tests.py | 17 +-
tests/db_engine_specs/hive_tests.py | 15 +-
tests/db_engine_specs/presto_tests.py | 14 +
tests/event_logger_tests.py | 5 +-
tests/fixtures/importexport.py | 306 +-
tests/fixtures/query_context.py | 73 +
tests/queries/saved_queries/api_tests.py | 16 +-
tests/reports/api_tests.py | 1 +
tests/security/migrate_roles_tests.py | 237 +
tests/security_tests.py | 29 +-
tests/sql_parse_tests.py | 33 +-
tests/sql_validator_tests.py | 31 +-
tests/superset_test_config.py | 2 +-
237 files changed, 9275 insertions(+), 3099 deletions(-)
delete mode 100755 RELEASING/make_docs.sh
delete mode 100755 RELEASING/make_docs_entrypoint.sh
copy RELEASING/Dockerfile.make_tarball => docker/docker-frontend.sh (81%)
mode change 100644 => 100755
create mode 100644 superset-frontend/images/empty-query.svg
copy superset-frontend/images/icons/{grid.svg => exclamation.svg} (65%)
create mode 100644 superset-frontend/images/icons/slack.svg
rename superset-frontend/spec/{javascripts/dashboard => }/fixtures/mockChartQueries.js (97%)
rename superset-frontend/spec/{javascripts/dashboard => }/fixtures/mockDashboardData.js (100%)
rename superset-frontend/spec/{javascripts/dashboard => }/fixtures/mockDashboardFilters.js (100%)
rename superset-frontend/spec/{javascripts/dashboard => }/fixtures/mockDashboardInfo.js (100%)
rename superset-frontend/spec/{javascripts/dashboard => }/fixtures/mockDashboardLayout.js (100%)
rename superset-frontend/spec/{javascripts/dashboard => }/fixtures/mockDashboardState.js (100%)
rename superset-frontend/spec/{javascripts/dashboard => }/fixtures/mockSliceEntities.js (98%)
rename superset-frontend/spec/{javascripts/dashboard => }/fixtures/mockState.js (90%)
rename superset-frontend/spec/{javascripts/dashboard => }/fixtures/mockStore.js (100%)
rename superset-frontend/spec/{javascripts/dashboard => }/helpers/WithDragDropContext.jsx (100%)
copy superset-frontend/{src/explore/propTypes/columnType.js => spec/helpers/setup.ts} (71%)
copy superset-frontend/{src/dashboard/util/logging/getLoadStatsPerTopLevelComponent.js => spec/helpers/testing-library.tsx} (55%)
create mode 100644 superset-frontend/spec/javascripts/components/Timer_spec.tsx
rename superset-frontend/spec/javascripts/explore/components/{ControlRow_spec.jsx => ControlRow_spec.tsx} (68%)
create mode 100644 superset-frontend/spec/javascripts/sqllab/SaveDatasetModal_spec.tsx
delete mode 100644 superset-frontend/spec/javascripts/sqllab/Timer_spec.jsx
create mode 100644 superset-frontend/spec/javascripts/views/CRUD/alert/AlertList_spec.jsx
create mode 100644 superset-frontend/src/SqlLab/components/SaveDatasetModal.tsx
create mode 100644 superset-frontend/src/api/dataset.ts
copy superset-frontend/src/{components/Loading.tsx => common/components/Badge.tsx} (59%)
copy superset-frontend/src/common/components/{Popover.tsx => Fade.tsx} (76%)
create mode 100644 superset-frontend/src/common/components/ProgressBar.tsx
copy superset-frontend/src/{SqlLab/components/TabStatusIcon.jsx => common/components/Switch.tsx} (69%)
copy superset-frontend/src/{SqlLab/components/TabStatusIcon.jsx => components/ListView/Filters/Base.ts} (66%)
create mode 100644 superset-frontend/src/components/ListView/Filters/DateRange.tsx
create mode 100644 superset-frontend/src/components/ListView/Filters/Search.tsx
rename superset-frontend/src/components/ListView/{Filters.tsx => Filters/Select.tsx} (56%)
create mode 100644 superset-frontend/src/components/ListView/Filters/index.tsx
delete mode 100644 superset-frontend/src/dashboard/components/DeleteComponentModal.jsx
create mode 100644 superset-frontend/src/explore/components/DataTableControl.tsx
create mode 100644 superset-frontend/src/explore/components/DataTablesPane.tsx
rename superset-frontend/src/setup/{setupClient.js => setupClient.ts} (83%)
rename superset-frontend/src/setup/{setupFormatters.js => setupFormatters.ts} (100%)
rename superset-frontend/src/setup/{setupPluginsExtra.js => setupPluginsExtra.ts} (100%)
create mode 100644 superset-frontend/src/utils/copy.ts
rename superset-frontend/src/utils/{errorMessages.js => errorMessages.ts} (100%)
create mode 100644 superset-frontend/src/views/CRUD/alert/AlertList.tsx
copy superset-frontend/src/views/CRUD/{annotation => alert}/types.ts (76%)
copy {tests/security => superset/migrations/shared}/__init__.py (100%)
create mode 100644 superset/migrations/shared/security_converge.py
copy superset/migrations/{versions/72428d1ea401_add_tmp_schema_name_to_the_query_object.py => shared/utils.py} (54%)
copy superset/migrations/versions/{30bb17c0dc76_.py => 811494c0cc23_remove_path_from_logs.py} (64%)
create mode 100644 superset/migrations/versions/8ee129739cf9_security_converge_css_templates.py
create mode 100644 superset/migrations/versions/e38177dbf641_security_converge_saved_queries.py
create mode 100644 superset/sql_validators/postgres.py
create mode 100644 tests/annotation_layers/fixtures.py
create mode 100644 tests/security/migrate_roles_tests.py
[incubator-superset] 01/01: WIP
Posted by be...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
beto pushed a commit to branch PISQL-2
in repository https://gitbox.apache.org/repos/asf/incubator-superset.git
commit 990ca8a414d177c5c08e3e4ff57f3d4d20488133
Author: Beto Dealmeida <ro...@dealmeida.net>
AuthorDate: Mon Dec 7 17:05:12 2020 -0800
WIP
---
.../components/ImportModal/ImportModal.test.tsx | 22 +--
.../src/chart/components/ImportModal/index.tsx | 186 +++++++++++++++++++++
.../components/ImportModal/ImportModal.test.tsx | 2 +-
.../src/views/CRUD/chart/ChartList.tsx | 36 +++-
superset-frontend/src/views/CRUD/chart/types.ts | 27 +++
superset/charts/api.py | 8 +-
superset/charts/commands/importers/dispatcher.py | 4 +-
superset/charts/commands/importers/v1/__init__.py | 18 ++
superset/charts/schemas.py | 2 +
superset/datasets/schemas.py | 2 +-
tests/fixtures/importexport.py | 2 +
11 files changed, 293 insertions(+), 16 deletions(-)
diff --git a/superset-frontend/src/datasource/components/ImportModal/ImportModal.test.tsx b/superset-frontend/src/chart/components/ImportModal/ImportModal.test.tsx
similarity index 83%
copy from superset-frontend/src/datasource/components/ImportModal/ImportModal.test.tsx
copy to superset-frontend/src/chart/components/ImportModal/ImportModal.test.tsx
index d6e49d5..ddcf44e 100644
--- a/superset-frontend/src/datasource/components/ImportModal/ImportModal.test.tsx
+++ b/superset-frontend/src/chart/components/ImportModal/ImportModal.test.tsx
@@ -22,7 +22,7 @@ import configureStore from 'redux-mock-store';
import { styledMount as mount } from 'spec/helpers/theming';
import { ReactWrapper } from 'enzyme';
-import ImportDatasetModal from 'src/datasource/components/ImportModal';
+import ImportChartModal from 'src/chart/components/ImportModal';
import Modal from 'src/common/components/Modal';
const mockStore = configureStore([thunk]);
@@ -31,16 +31,16 @@ const store = mockStore({});
const requiredProps = {
addDangerToast: () => {},
addSuccessToast: () => {},
- onDatasetImport: () => {},
+ onChartImport: () => {},
show: true,
onHide: () => {},
};
-describe('ImportDatasetModal', () => {
+describe('ImportChartModal', () => {
let wrapper: ReactWrapper;
beforeEach(() => {
- wrapper = mount(<ImportDatasetModal {...requiredProps} />, {
+ wrapper = mount(<ImportChartModal {...requiredProps} />, {
context: { store },
});
});
@@ -50,15 +50,15 @@ describe('ImportDatasetModal', () => {
});
it('renders', () => {
- expect(wrapper.find(ImportDatasetModal)).toExist();
+ expect(wrapper.find(ImportChartModal)).toExist();
});
it('renders a Modal', () => {
expect(wrapper.find(Modal)).toExist();
});
- it('renders "Import Dataset" header', () => {
- expect(wrapper.find('h4').text()).toEqual('Import Dataset');
+ it('renders "Import Chart" header', () => {
+ expect(wrapper.find('h4').text()).toEqual('Import Chart');
});
it('renders a label and a file input field', () => {
@@ -67,7 +67,7 @@ describe('ImportDatasetModal', () => {
});
it('should attach the label to the input field', () => {
- const id = 'datasetFile';
+ const id = 'chartFile';
expect(wrapper.find('label').prop('htmlFor')).toBe(id);
expect(wrapper.find('input').prop('id')).toBe(id);
});
@@ -83,7 +83,7 @@ describe('ImportDatasetModal', () => {
});
it('should render the import button enabled when a file is selected', () => {
- const file = new File([new ArrayBuffer(1)], 'dataset_export.zip');
+ const file = new File([new ArrayBuffer(1)], 'chart_export.zip');
wrapper.find('input').simulate('change', { target: { files: [file] } });
expect(wrapper.find('button[children="Import"]').prop('disabled')).toBe(
@@ -93,9 +93,9 @@ describe('ImportDatasetModal', () => {
it('should render password fields when needed for import', () => {
const wrapperWithPasswords = mount(
- <ImportDatasetModal
+ <ImportChartModal
{...requiredProps}
- passwordFields={['datasets/examples.yaml']}
+ passwordFields={['databases/examples.yaml']}
/>,
{
context: { store },
diff --git a/superset-frontend/src/chart/components/ImportModal/index.tsx b/superset-frontend/src/chart/components/ImportModal/index.tsx
new file mode 100644
index 0000000..328af18
--- /dev/null
+++ b/superset-frontend/src/chart/components/ImportModal/index.tsx
@@ -0,0 +1,186 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+import React, { FunctionComponent, useEffect, useRef, useState } from 'react';
+import { t } from '@superset-ui/core';
+
+import Modal from 'src/common/components/Modal';
+import {
+ StyledIcon,
+ StyledInputContainer,
+} from 'src/views/CRUD/data/database/DatabaseModal';
+import { useImportResource } from 'src/views/CRUD/hooks';
+import { ChartObject } from 'src/views/CRUD/chart/types';
+
+export interface ImportChartModalProps {
+ addDangerToast: (msg: string) => void;
+ addSuccessToast: (msg: string) => void;
+ onChartImport: () => void;
+ show: boolean;
+ onHide: () => void;
+ passwordFields?: string[];
+ setPasswordFields?: (passwordFields: string[]) => void;
+}
+
+const ImportChartModal: FunctionComponent<ImportChartModalProps> = ({
+ addDangerToast,
+ addSuccessToast,
+ onChartImport,
+ show,
+ onHide,
+ passwordFields = [],
+ setPasswordFields = () => {},
+}) => {
+ const [uploadFile, setUploadFile] = useState<File | null>(null);
+ const [isHidden, setIsHidden] = useState<boolean>(true);
+ const [passwords, setPasswords] = useState<Record<string, string>>({});
+ const fileInputRef = useRef<HTMLInputElement>(null);
+
+ const clearModal = () => {
+ setUploadFile(null);
+ setPasswordFields([]);
+ setPasswords({});
+ if (fileInputRef && fileInputRef.current) {
+ fileInputRef.current.value = '';
+ }
+ };
+
+ const handleErrorMsg = (msg: string) => {
+ clearModal();
+ addDangerToast(msg);
+ };
+
+ const {
+ state: { passwordsNeeded },
+ importResource,
+ } = useImportResource<ChartObject>('chart', t('chart'), handleErrorMsg);
+
+ useEffect(() => {
+ setPasswordFields(passwordsNeeded);
+ }, [passwordsNeeded]);
+
+ // Functions
+ const hide = () => {
+ setIsHidden(true);
+ onHide();
+ };
+
+ const onUpload = () => {
+ if (uploadFile === null) {
+ return;
+ }
+
+ importResource(uploadFile, passwords).then(result => {
+ if (result) {
+ addSuccessToast(t('The charts have been imported'));
+ clearModal();
+ onChartImport();
+ }
+ });
+ };
+
+ const changeFile = (event: React.ChangeEvent<HTMLInputElement>) => {
+ const { files } = event.target as HTMLInputElement;
+ setUploadFile((files && files[0]) || null);
+ };
+
+ const renderPasswordFields = () => {
+ if (passwordFields.length === 0) {
+ return null;
+ }
+
+ return (
+ <>
+ <h5>Database passwords</h5>
+ <StyledInputContainer>
+ <div className="helper">
+ {t(
+ 'The passwords for the databases below are needed in order to ' +
+ 'import them together with the charts. Please note that the ' +
+ '"Secure Extra" and "Certificate" sections of ' +
+ 'the database configuration are not present in export files, and ' +
+ 'should be added manually after the import if they are needed.',
+ )}
+ </div>
+ </StyledInputContainer>
+ {passwordFields.map(fileName => (
+ <StyledInputContainer key={`password-for-${fileName}`}>
+ <div className="control-label">
+ {fileName}
+ <span className="required">*</span>
+ </div>
+ <input
+ name={`password-${fileName}`}
+ autoComplete="off"
+ type="password"
+ value={passwords[fileName]}
+ onChange={event =>
+ setPasswords({ ...passwords, [fileName]: event.target.value })
+ }
+ />
+ </StyledInputContainer>
+ ))}
+ </>
+ );
+ };
+
+ // Show/hide
+ if (isHidden && show) {
+ setIsHidden(false);
+ }
+
+ return (
+ <Modal
+ name="chart"
+ className="chart-modal"
+ disablePrimaryButton={uploadFile === null}
+ onHandledPrimaryAction={onUpload}
+ onHide={hide}
+ primaryButtonName={t('Import')}
+ width="750px"
+ show={show}
+ title={
+ <h4>
+ <StyledIcon name="nav-charts" />
+ {t('Import Chart')}
+ </h4>
+ }
+ >
+ <StyledInputContainer>
+ <div className="control-label">
+ <label htmlFor="chartFile">
+ {t('File')}
+ <span className="required">*</span>
+ </label>
+ </div>
+ <input
+ ref={fileInputRef}
+ data-test="chart-file-input"
+ name="chartFile"
+ id="chartFile"
+ type="file"
+ accept=".yaml,.json,.yml,.zip"
+ onChange={changeFile}
+ />
+ </StyledInputContainer>
+ {renderPasswordFields()}
+ </Modal>
+ );
+};
+
+export default ImportChartModal;
diff --git a/superset-frontend/src/datasource/components/ImportModal/ImportModal.test.tsx b/superset-frontend/src/datasource/components/ImportModal/ImportModal.test.tsx
index d6e49d5..7087129 100644
--- a/superset-frontend/src/datasource/components/ImportModal/ImportModal.test.tsx
+++ b/superset-frontend/src/datasource/components/ImportModal/ImportModal.test.tsx
@@ -95,7 +95,7 @@ describe('ImportDatasetModal', () => {
const wrapperWithPasswords = mount(
<ImportDatasetModal
{...requiredProps}
- passwordFields={['datasets/examples.yaml']}
+ passwordFields={['databases/examples.yaml']}
/>,
{
context: { store },
diff --git a/superset-frontend/src/views/CRUD/chart/ChartList.tsx b/superset-frontend/src/views/CRUD/chart/ChartList.tsx
index e7cf14a..ea787e1 100644
--- a/superset-frontend/src/views/CRUD/chart/ChartList.tsx
+++ b/superset-frontend/src/views/CRUD/chart/ChartList.tsx
@@ -17,7 +17,7 @@
* under the License.
*/
import { SupersetClient, getChartMetadataRegistry, t } from '@superset-ui/core';
-import React, { useMemo } from 'react';
+import React, { useMemo, useState } from 'react';
import rison from 'rison';
import { uniqBy } from 'lodash';
import { isFeatureEnabled, FeatureFlag } from 'src/featureFlags';
@@ -43,6 +43,7 @@ import ListView, {
} from 'src/components/ListView';
import withToasts from 'src/messageToasts/enhancers/withToasts';
import PropertiesModal from 'src/explore/components/PropertiesModal';
+import ImportChartModal from 'src/chart/components/ImportModal/index';
import Chart from 'src/types/Chart';
import TooltipWrapper from 'src/components/TooltipWrapper';
import ChartCard from './ChartCard';
@@ -124,6 +125,22 @@ function ChartList(props: ChartListProps) {
closeChartEditModal,
} = useChartEditModal(setCharts, charts);
+ const [importingChart, showImportModal] = useState<boolean>(false);
+ const [passwordFields, setPasswordFields] = useState<string[]>([]);
+
+ function openChartImportModal() {
+ showImportModal(true);
+ }
+
+ function closeChartImportModal() {
+ showImportModal(false);
+ }
+
+ const handleChartImport = () => {
+ showImportModal(false);
+ refreshData();
+ };
+
const canCreate = hasPerm('can_add');
const canEdit = hasPerm('can_edit');
const canDelete = hasPerm('can_delete');
@@ -482,6 +499,13 @@ function ChartList(props: ChartListProps) {
},
});
}
+ if (isFeatureEnabled(FeatureFlag.VERSIONED_EXPORT)) {
+ subMenuButtons.push({
+ name: <Icon name="import" />,
+ buttonStyle: 'link',
+ onClick: openChartImportModal,
+ });
+ }
return (
<>
<SubMenu name={t('Charts')} buttons={subMenuButtons} />
@@ -541,6 +565,16 @@ function ChartList(props: ChartListProps) {
);
}}
</ConfirmStatusChange>
+
+ <ImportChartModal
+ show={importingChart}
+ onHide={closeChartImportModal}
+ addDangerToast={props.addDangerToast}
+ addSuccessToast={props.addSuccessToast}
+ onChartImport={handleChartImport}
+ passwordFields={passwordFields}
+ setPasswordFields={setPasswordFields}
+ />
</>
);
}
diff --git a/superset-frontend/src/views/CRUD/chart/types.ts b/superset-frontend/src/views/CRUD/chart/types.ts
new file mode 100644
index 0000000..209d009
--- /dev/null
+++ b/superset-frontend/src/views/CRUD/chart/types.ts
@@ -0,0 +1,27 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+export type ChartObject = {
+ slice_name?: string;
+ description?: string;
+ viz_type?: string;
+ params?: string;
+ cache_timeout?: number;
+ datasource_id?: number;
+ datasource_type?: number;
+};
diff --git a/superset/charts/api.py b/superset/charts/api.py
index 262d2cf..59a8dfc 100644
--- a/superset/charts/api.py
+++ b/superset/charts/api.py
@@ -878,7 +878,13 @@ class ChartRestApi(BaseSupersetModelRestApi):
for file_name in bundle.namelist()
}
- command = ImportChartsCommand(contents)
+ passwords = (
+ json.loads(request.form["passwords"])
+ if "passwords" in request.form
+ else None
+ )
+
+ command = ImportChartsCommand(contents, passwords=passwords)
try:
command.run()
return self.response(200, message="OK")
diff --git a/superset/charts/commands/importers/dispatcher.py b/superset/charts/commands/importers/dispatcher.py
index 2098ea7..ff34889 100644
--- a/superset/charts/commands/importers/dispatcher.py
+++ b/superset/charts/commands/importers/dispatcher.py
@@ -43,12 +43,14 @@ class ImportChartsCommand(BaseCommand):
# pylint: disable=unused-argument
def __init__(self, contents: Dict[str, str], *args: Any, **kwargs: Any):
self.contents = contents
+ self.args = args
+ self.kwargs = kwargs
def run(self) -> None:
# iterate over all commands until we find a version that can
# handle the contents
for version in command_versions:
- command = version(self.contents)
+ command = version(self.contents, *self.args, **self.kwargs)
try:
command.run()
return
diff --git a/superset/charts/commands/importers/v1/__init__.py b/superset/charts/commands/importers/v1/__init__.py
index 4aed3fa..69840f6 100644
--- a/superset/charts/commands/importers/v1/__init__.py
+++ b/superset/charts/commands/importers/v1/__init__.py
@@ -36,6 +36,7 @@ from superset.databases.commands.importers.v1.utils import import_database
from superset.databases.schemas import ImportV1DatabaseSchema
from superset.datasets.commands.importers.v1.utils import import_dataset
from superset.datasets.schemas import ImportV1DatasetSchema
+from superset.models.core import Database
from superset.models.slice import Slice
schemas: Dict[str, Schema] = {
@@ -52,6 +53,7 @@ class ImportChartsCommand(BaseCommand):
# pylint: disable=unused-argument
def __init__(self, contents: Dict[str, str], *args: Any, **kwargs: Any):
self.contents = contents
+ self.passwords: Dict[str, str] = kwargs.get("passwords") or {}
self._configs: Dict[str, Any] = {}
def _import_bundle(self, session: Session) -> None:
@@ -113,6 +115,14 @@ class ImportChartsCommand(BaseCommand):
def validate(self) -> None:
exceptions: List[ValidationError] = []
+ # load existing databases so we can apply the password validation
+ db_passwords = {
+ str(uuid): password
+ for uuid, password in db.session.query(
+ Database.uuid, Database.password
+ ).all()
+ }
+
# verify that the metadata file is present and valid
try:
metadata: Optional[Dict[str, str]] = load_metadata(self.contents)
@@ -120,12 +130,20 @@ class ImportChartsCommand(BaseCommand):
exceptions.append(exc)
metadata = None
+ # validate charts, datasets, and databases
for file_name, content in self.contents.items():
prefix = file_name.split("/")[0]
schema = schemas.get(f"{prefix}/")
if schema:
try:
config = load_yaml(file_name, content)
+
+ # populate passwords from the request or from existing DBs
+ if file_name in self.passwords:
+ config["password"] = self.passwords[file_name]
+ elif prefix == "databases" and config["uuid"] in db_passwords:
+ config["password"] = db_passwords[config["uuid"]]
+
schema.load(config)
self._configs[file_name] = config
except ValidationError as exc:
diff --git a/superset/charts/schemas.py b/superset/charts/schemas.py
index 5e346fa..347189a 100644
--- a/superset/charts/schemas.py
+++ b/superset/charts/schemas.py
@@ -1118,6 +1118,8 @@ class GetFavStarIdsSchema(Schema):
class ImportV1ChartSchema(Schema):
+ slice_name = fields.String(required=True)
+ viz_type = fields.String(required=True)
params = fields.Dict()
cache_timeout = fields.Integer(allow_none=True)
uuid = fields.UUID(required=True)
diff --git a/superset/datasets/schemas.py b/superset/datasets/schemas.py
index 373e49f..b8406eb 100644
--- a/superset/datasets/schemas.py
+++ b/superset/datasets/schemas.py
@@ -129,7 +129,7 @@ class ImportV1ColumnSchema(Schema):
verbose_name = fields.String(allow_none=True)
is_dttm = fields.Boolean()
is_active = fields.Boolean(allow_none=True)
- type = fields.String(required=True)
+ type = fields.String(allow_none=True)
groupby = fields.Boolean()
filterable = fields.Boolean()
expression = fields.String(allow_none=True)
diff --git a/tests/fixtures/importexport.py b/tests/fixtures/importexport.py
index 6383923..f0c35a1 100644
--- a/tests/fixtures/importexport.py
+++ b/tests/fixtures/importexport.py
@@ -404,6 +404,8 @@ dataset_config: Dict[str, Any] = {
}
chart_config: Dict[str, Any] = {
+ "slice_name": "Deck Path",
+ "viz_type": "deck_path",
"params": {
"color_picker": {"a": 1, "b": 135, "g": 122, "r": 0},
"datasource": "12__table",