You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by GitBox <gi...@apache.org> on 2022/12/01 15:56:36 UTC
[GitHub] [superset] cccs-RyanK opened a new pull request, #22304: feat: add Dataset tagging import/update api
cccs-RyanK opened a new pull request, #22304:
URL: https://github.com/apache/superset/pull/22304
<!---
Please write the PR title following the conventions at https://www.conventionalcommits.org/en/v1.0.0/
Example:
fix(dashboard): load charts correctly
-->
### SUMMARY
<!--- Describe the change below, including rationale and design decisions -->
This work is related to a [previous PR](https://github.com/apache/superset/pull/20892) which added the ability to tag datasets in the backend. This PR adds the support tobe able to add tags to datasets via the API.
Currently there is an [ongoing PR](https://github.com/apache/superset/pull/20876) to add tags to the front-end.
It is behind the TAGGING_SYSTEM feature flag
### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
<!--- Skip this if not applicable -->
### TESTING INSTRUCTIONS
<!--- Required! What steps can be taken to manually verify the changes? -->
Tests added to tests/integration_tests/datasets/api_tests.py
### ADDITIONAL INFORMATION
<!--- Check any relevant boxes with "x" -->
<!--- HINT: Include "Fixes #nnn" if you are fixing an existing issue -->
- [ ] Has associated issue:
- [x] Required feature flags: TAGGING_SYSTEM
- [ ] Changes UI
- [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351))
- [ ] Migration is atomic, supports rollback & is backwards-compatible
- [ ] Confirm DB migration upgrade and downgrade tested
- [ ] Runtime estimates and downtime expectations provided
- [x] Introduces new feature or API
- [ ] Removes existing feature or API
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] hughhhh commented on a diff in pull request #22304: feat: add Dataset tagging import/update api
Posted by GitBox <gi...@apache.org>.
hughhhh commented on code in PR #22304:
URL: https://github.com/apache/superset/pull/22304#discussion_r1081860789
##########
superset/datasets/commands/importers/v1/utils.py:
##########
@@ -148,6 +156,26 @@ def import_dataset(
return dataset
+def extract_tags(config: Dict[str, Any]) -> Tuple[Dict[str, Any], Optional[List[str]]]:
+ tags = None
+ if "tags" in config.keys() and feature_flag_manager.is_feature_enabled(
+ "TAGGING_SYSTEM"
+ ):
+ tags = config.pop("tags")
+
+ return config, tags
+
+
+def import_tags(dataset: SqlaTable, existing: bool, tags: Optional[List[str]]) -> None:
Review Comment:
can we get unit test for this function
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] Antonio-RiveroMartnez commented on a diff in pull request #22304: feat: add Dataset tagging import/update api
Posted by GitBox <gi...@apache.org>.
Antonio-RiveroMartnez commented on code in PR #22304:
URL: https://github.com/apache/superset/pull/22304#discussion_r1081697416
##########
superset/datasets/dao.py:
##########
@@ -167,6 +169,11 @@ def update(
if "metrics" in properties:
cls.update_metrics(model, properties.pop("metrics"), commit=commit)
+ if "tags" in properties and feature_flag_manager.is_feature_enabled(
Review Comment:
same as above
##########
superset/datasets/commands/importers/v1/utils.py:
##########
@@ -148,6 +156,26 @@ def import_dataset(
return dataset
+def extract_tags(config: Dict[str, Any]) -> Tuple[Dict[str, Any], Optional[List[str]]]:
+ tags = None
+ if "tags" in config.keys() and feature_flag_manager.is_feature_enabled(
Review Comment:
I see a broader usage in our app of `is_feature_enabled` coming from this import directly `from superset import is_feature_enabled`, any reason why not using that instead?
##########
superset/datasets/commands/importers/v1/utils.py:
##########
@@ -148,6 +156,26 @@ def import_dataset(
return dataset
+def extract_tags(config: Dict[str, Any]) -> Tuple[Dict[str, Any], Optional[List[str]]]:
+ tags = None
+ if "tags" in config.keys() and feature_flag_manager.is_feature_enabled(
+ "TAGGING_SYSTEM"
+ ):
+ tags = config.pop("tags")
+
+ return config, tags
+
+
+def import_tags(dataset: SqlaTable, existing: bool, tags: Optional[List[str]]) -> None:
Review Comment:
Looking at this method, it's only doing something if `tags` has value in it, so why is `tags` `Optional` then? Can we just mark it as required and not call this method if tag if empty for example?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] hughhhh commented on a diff in pull request #22304: feat: add Dataset tagging import/update api
Posted by GitBox <gi...@apache.org>.
hughhhh commented on code in PR #22304:
URL: https://github.com/apache/superset/pull/22304#discussion_r1081859129
##########
superset/datasets/commands/importers/v1/utils.py:
##########
@@ -148,6 +156,26 @@ def import_dataset(
return dataset
+def extract_tags(config: Dict[str, Any]) -> Tuple[Dict[str, Any], Optional[List[str]]]:
Review Comment:
can we get a unit test for this util function
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] Antonio-RiveroMartnez commented on a diff in pull request #22304: feat: add Dataset tagging import/update api
Posted by GitBox <gi...@apache.org>.
Antonio-RiveroMartnez commented on code in PR #22304:
URL: https://github.com/apache/superset/pull/22304#discussion_r1081703792
##########
superset/datasets/commands/importers/v1/utils.py:
##########
@@ -148,6 +156,26 @@ def import_dataset(
return dataset
+def extract_tags(config: Dict[str, Any]) -> Tuple[Dict[str, Any], Optional[List[str]]]:
+ tags = None
+ if "tags" in config.keys() and feature_flag_manager.is_feature_enabled(
+ "TAGGING_SYSTEM"
+ ):
+ tags = config.pop("tags")
+
+ return config, tags
+
+
+def import_tags(dataset: SqlaTable, existing: bool, tags: Optional[List[str]]) -> None:
Review Comment:
Looking at this method, it's only doing something if `tags` has value in it, so why is `tags` `Optional` then? Can we just mark it as required and not call this method if tag if empty for example? or do you see a reason to keep it as is?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat: add Dataset tagging import/update api [superset]
Posted by "cccs-RyanK (via GitHub)" <gi...@apache.org>.
cccs-RyanK commented on PR #22304:
URL: https://github.com/apache/superset/pull/22304#issuecomment-2682582568
> Is there any interest in picking this back up, or should it be closed out?
Hey ! sorry for the very late reply. This particular issue has tumbled down our priority list, but as of right now we still do intend on continuing it as it is still a requirement from our users and I think it is a valuable feature for Superset in general. There is no immediate plan to work on this so I am going to close this with the intention of reopening it down the line.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat: add Dataset tagging import/update api [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on PR #22304:
URL: https://github.com/apache/superset/pull/22304#issuecomment-2532623836
Is there any interest in picking this back up, or should it be closed out?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] hughhhh commented on a diff in pull request #22304: feat: add Dataset tagging import/update api
Posted by GitBox <gi...@apache.org>.
hughhhh commented on code in PR #22304:
URL: https://github.com/apache/superset/pull/22304#discussion_r1081861702
##########
superset/tags/utils.py:
##########
@@ -0,0 +1,145 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from typing import List, Optional
+
+from sqlalchemy import Enum
+
+from superset import db
+from superset.tags.exceptions import CustomTagNameValidationError
+from superset.tags.models import get_tag, ObjectTypes, Tag, TaggedObject, TagTypes
+
+
+def validate_custom_tags(
Review Comment:
test?
##########
superset/tags/utils.py:
##########
@@ -0,0 +1,145 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from typing import List, Optional
+
+from sqlalchemy import Enum
+
+from superset import db
+from superset.tags.exceptions import CustomTagNameValidationError
+from superset.tags.models import get_tag, ObjectTypes, Tag, TaggedObject, TagTypes
+
+
+def validate_custom_tags(
+ tags: List[str],
+) -> bool:
+ """
+ Validates custom tags.
+ It is important that custom tags do not have ":" in them because
+ this character is used specifically for the other types of tags
+ tags:
+ list of names of tags to validate
+ """
+ for tag in tags:
+ if ":" in tag:
+ return False
+ return True
+
+
+def add_custom_object_tags(
+ tags: List[str],
Review Comment:
test?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] codecov[bot] commented on pull request #22304: feat: add Dataset tagging import/update api
Posted by GitBox <gi...@apache.org>.
codecov[bot] commented on PR #22304:
URL: https://github.com/apache/superset/pull/22304#issuecomment-1333995571
# [Codecov](https://codecov.io/gh/apache/superset/pull/22304?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#22304](https://codecov.io/gh/apache/superset/pull/22304?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (28fff40) into [master](https://codecov.io/gh/apache/superset/commit/a642d126f8019d8f96cc206abfeda7ddc19eda7f?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (a642d12) will **decrease** coverage by `11.35%`.
> The diff coverage is `47.42%`.
> :exclamation: Current head 28fff40 differs from pull request most recent head 0b659cc. Consider uploading reports for the commit 0b659cc to get more accurate results
```diff
@@ Coverage Diff @@
## master #22304 +/- ##
===========================================
- Coverage 66.86% 55.50% -11.36%
===========================================
Files 1841 1848 +7
Lines 70220 70369 +149
Branches 7670 7688 +18
===========================================
- Hits 46950 39060 -7890
- Misses 21282 29315 +8033
- Partials 1988 1994 +6
```
| Flag | Coverage Δ | |
|---|---|---|
| hive | `?` | |
| mysql | `?` | |
| postgres | `?` | |
| presto | `52.40% <36.36%> (-0.06%)` | :arrow_down: |
| python | `57.38% <41.81%> (-23.96%)` | :arrow_down: |
| sqlite | `?` | |
| unit | `50.83% <41.81%> (-0.04%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/superset/pull/22304?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...packages/superset-ui-core/src/query/types/Query.ts](https://codecov.io/gh/apache/superset/pull/22304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGFja2FnZXMvc3VwZXJzZXQtdWktY29yZS9zcmMvcXVlcnkvdHlwZXMvUXVlcnkudHM=) | `100.00% <ø> (ø)` | |
| [...ugins/plugin-chart-echarts/src/Radar/buildQuery.ts](https://codecov.io/gh/apache/superset/pull/22304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGx1Z2lucy9wbHVnaW4tY2hhcnQtZWNoYXJ0cy9zcmMvUmFkYXIvYnVpbGRRdWVyeS50cw==) | `0.00% <0.00%> (ø)` | |
| [.../components/Chart/DrillDetail/DrillDetailModal.tsx](https://codecov.io/gh/apache/superset/pull/22304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvc3JjL2NvbXBvbmVudHMvQ2hhcnQvRHJpbGxEZXRhaWwvRHJpbGxEZXRhaWxNb2RhbC50c3g=) | `92.30% <ø> (ø)` | |
| [...nts/Chart/DrillDetail/DrillDetailTableControls.tsx](https://codecov.io/gh/apache/superset/pull/22304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvc3JjL2NvbXBvbmVudHMvQ2hhcnQvRHJpbGxEZXRhaWwvRHJpbGxEZXRhaWxUYWJsZUNvbnRyb2xzLnRzeA==) | `94.11% <ø> (ø)` | |
| [...ts/Table/header-renderers/HeaderWithRadioGroup.tsx](https://codecov.io/gh/apache/superset/pull/22304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvc3JjL2NvbXBvbmVudHMvVGFibGUvaGVhZGVyLXJlbmRlcmVycy9IZWFkZXJXaXRoUmFkaW9Hcm91cC50c3g=) | `0.00% <0.00%> (ø)` | |
| [...d/components/DashboardBuilder/DashboardBuilder.tsx](https://codecov.io/gh/apache/superset/pull/22304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvc3JjL2Rhc2hib2FyZC9jb21wb25lbnRzL0Rhc2hib2FyZEJ1aWxkZXIvRGFzaGJvYXJkQnVpbGRlci50c3g=) | `74.28% <0.00%> (-0.96%)` | :arrow_down: |
| [.../src/dashboard/components/gridComponents/Chart.jsx](https://codecov.io/gh/apache/superset/pull/22304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvc3JjL2Rhc2hib2FyZC9jb21wb25lbnRzL2dyaWRDb21wb25lbnRzL0NoYXJ0LmpzeA==) | `53.98% <ø> (ø)` | |
| [...Filters/FilterBar/FilterControls/FilterControl.tsx](https://codecov.io/gh/apache/superset/pull/22304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvc3JjL2Rhc2hib2FyZC9jb21wb25lbnRzL25hdGl2ZUZpbHRlcnMvRmlsdGVyQmFyL0ZpbHRlckNvbnRyb2xzL0ZpbHRlckNvbnRyb2wudHN4) | `29.03% <ø> (ø)` | |
| [...Filters/FilterBar/FilterControls/FilterDivider.tsx](https://codecov.io/gh/apache/superset/pull/22304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvc3JjL2Rhc2hib2FyZC9jb21wb25lbnRzL25hdGl2ZUZpbHRlcnMvRmlsdGVyQmFyL0ZpbHRlckNvbnRyb2xzL0ZpbHRlckRpdmlkZXIudHN4) | `84.00% <0.00%> (+2.51%)` | :arrow_up: |
| [...Filters/FilterBar/FiltersDropdownContent/index.tsx](https://codecov.io/gh/apache/superset/pull/22304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvc3JjL2Rhc2hib2FyZC9jb21wb25lbnRzL25hdGl2ZUZpbHRlcnMvRmlsdGVyQmFyL0ZpbHRlcnNEcm9wZG93bkNvbnRlbnQvaW5kZXgudHN4) | `25.00% <ø> (ø)` | |
| ... and [322 more](https://codecov.io/gh/apache/superset/pull/22304/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat: add Dataset tagging import/update api [superset]
Posted by "geido (via GitHub)" <gi...@apache.org>.
geido commented on PR #22304:
URL: https://github.com/apache/superset/pull/22304#issuecomment-1986241835
Hey @cccs-RyanK I moved this to a draft as it seems it requires a rebase.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat: add Dataset tagging import/update api [superset]
Posted by "cccs-RyanK (via GitHub)" <gi...@apache.org>.
cccs-RyanK closed pull request #22304: feat: add Dataset tagging import/update api
URL: https://github.com/apache/superset/pull/22304
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org