You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by GitBox <gi...@apache.org> on 2022/07/26 01:33:58 UTC
[GitHub] [superset] hughhhh opened a new pull request, #20862: fix: overwrite update override columns on PUT /dataset
hughhhh opened a new pull request, #20862:
URL: https://github.com/apache/superset/pull/20862
<!---
Please write the PR title following the conventions at https://www.conventionalcommits.org/en/v1.0.0/
Example:
fix(dashboard): load charts correctly
-->
### SUMMARY
<!--- Describe the change below, including rationale and design decisions -->
### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
<!--- Skip this if not applicable -->
### TESTING INSTRUCTIONS
<!--- Required! What steps can be taken to manually verify the changes? -->
### ADDITIONAL INFORMATION
<!--- Check any relevant boxes with "x" -->
<!--- HINT: Include "Fixes #nnn" if you are fixing an existing issue -->
- [ ] Has associated issue:
- [ ] Required feature flags:
- [ ] Changes UI
- [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351))
- [ ] Migration is atomic, supports rollback & is backwards-compatible
- [ ] Confirm DB migration upgrade and downgrade tested
- [ ] Runtime estimates and downtime expectations provided
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] eschutho commented on a diff in pull request #20862: fix: overwrite update override columns on PUT /dataset
Posted by GitBox <gi...@apache.org>.
eschutho commented on code in PR #20862:
URL: https://github.com/apache/superset/pull/20862#discussion_r932764925
##########
superset/datasets/dao.py:
##########
@@ -147,14 +147,22 @@ def validate_metrics_uniqueness(dataset_id: int, metrics_names: List[str]) -> bo
@classmethod
def update(
- cls, model: SqlaTable, properties: Dict[str, Any], commit: bool = True
+ cls,
+ model: SqlaTable,
+ properties: Dict[str, Any],
+ commit: bool = True,
) -> Optional[SqlaTable]:
"""
Updates a Dataset model on the metadata DB
"""
if "columns" in properties:
- cls.update_columns(model, properties.pop("columns"), commit=commit)
+ cls.update_columns(
+ model,
+ properties.pop("columns"),
+ commit=commit,
+ override_columns=bool(properties.get("override_columns")),
+ )
if "metrics" in properties:
cls.update_metrics(model, properties.pop("metrics"), commit=commit)
Review Comment:
for put /dataset, does override_columns literally mean just the columns or should we be overriding the metrics, too?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] hughhhh commented on a diff in pull request #20862: fix: overwrite update override columns on PUT /dataset
Posted by GitBox <gi...@apache.org>.
hughhhh commented on code in PR #20862:
URL: https://github.com/apache/superset/pull/20862#discussion_r933633325
##########
superset/datasets/dao.py:
##########
@@ -147,14 +147,22 @@ def validate_metrics_uniqueness(dataset_id: int, metrics_names: List[str]) -> bo
@classmethod
def update(
- cls, model: SqlaTable, properties: Dict[str, Any], commit: bool = True
+ cls,
+ model: SqlaTable,
+ properties: Dict[str, Any],
+ commit: bool = True,
) -> Optional[SqlaTable]:
"""
Updates a Dataset model on the metadata DB
"""
if "columns" in properties:
- cls.update_columns(model, properties.pop("columns"), commit=commit)
+ cls.update_columns(
+ model,
+ properties.pop("columns"),
+ commit=commit,
+ override_columns=bool(properties.get("override_columns")),
+ )
if "metrics" in properties:
cls.update_metrics(model, properties.pop("metrics"), commit=commit)
Review Comment:
from sqllab, we don't allow users to create metrics so if you look at the `update_metrics` function we end up just deleting all the rows since `properties["metrics"]` is always None from that context.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] hughhhh merged pull request #20862: fix: overwrite update override columns on PUT /dataset
Posted by GitBox <gi...@apache.org>.
hughhhh merged PR #20862:
URL: https://github.com/apache/superset/pull/20862
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] codecov[bot] commented on pull request #20862: fix: overwrite update override columns on PUT /dataset
Posted by GitBox <gi...@apache.org>.
codecov[bot] commented on PR #20862:
URL: https://github.com/apache/superset/pull/20862#issuecomment-1194873324
# [Codecov](https://codecov.io/gh/apache/superset/pull/20862?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#20862](https://codecov.io/gh/apache/superset/pull/20862?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (52d4ec8) into [master](https://codecov.io/gh/apache/superset/commit/e3c6380258a46467ac8e09e64db60aad46bc4655?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (e3c6380) will **decrease** coverage by `11.68%`.
> The diff coverage is `0.00%`.
```diff
@@ Coverage Diff @@
## master #20862 +/- ##
===========================================
- Coverage 66.30% 54.62% -11.69%
===========================================
Files 1757 1757
Lines 66860 66874 +14
Branches 7077 7077
===========================================
- Hits 44332 36527 -7805
- Misses 20719 28538 +7819
Partials 1809 1809
```
| Flag | Coverage Δ | |
|---|---|---|
| hive | `?` | |
| mysql | `?` | |
| postgres | `?` | |
| presto | `53.11% <0.00%> (-0.01%)` | :arrow_down: |
| python | `57.45% <0.00%> (-24.14%)` | :arrow_down: |
| sqlite | `?` | |
| unit | `50.25% <0.00%> (-0.01%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/superset/pull/20862?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [superset/datasets/commands/update.py](https://codecov.io/gh/apache/superset/pull/20862/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGF0YXNldHMvY29tbWFuZHMvdXBkYXRlLnB5) | `25.00% <0.00%> (-68.98%)` | :arrow_down: |
| [superset/datasets/dao.py](https://codecov.io/gh/apache/superset/pull/20862/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGF0YXNldHMvZGFvLnB5) | `46.42% <0.00%> (-47.69%)` | :arrow_down: |
| [superset/utils/dashboard\_import\_export.py](https://codecov.io/gh/apache/superset/pull/20862/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdXRpbHMvZGFzaGJvYXJkX2ltcG9ydF9leHBvcnQucHk=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [superset/key\_value/commands/update.py](https://codecov.io/gh/apache/superset/pull/20862/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL3VwZGF0ZS5weQ==) | `0.00% <0.00%> (-88.89%)` | :arrow_down: |
| [superset/key\_value/commands/delete.py](https://codecov.io/gh/apache/superset/pull/20862/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL2RlbGV0ZS5weQ==) | `0.00% <0.00%> (-85.30%)` | :arrow_down: |
| [superset/db\_engines/hive.py](https://codecov.io/gh/apache/superset/pull/20862/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGJfZW5naW5lcy9oaXZlLnB5) | `0.00% <0.00%> (-85.19%)` | :arrow_down: |
| [superset/key\_value/commands/delete\_expired.py](https://codecov.io/gh/apache/superset/pull/20862/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL2RlbGV0ZV9leHBpcmVkLnB5) | `0.00% <0.00%> (-80.77%)` | :arrow_down: |
| [superset/dashboards/commands/importers/v0.py](https://codecov.io/gh/apache/superset/pull/20862/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGFzaGJvYXJkcy9jb21tYW5kcy9pbXBvcnRlcnMvdjAucHk=) | `15.62% <0.00%> (-76.25%)` | :arrow_down: |
| [superset/datasets/commands/create.py](https://codecov.io/gh/apache/superset/pull/20862/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGF0YXNldHMvY29tbWFuZHMvY3JlYXRlLnB5) | `29.41% <0.00%> (-68.63%)` | :arrow_down: |
| ... and [281 more](https://codecov.io/gh/apache/superset/pull/20862/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] betodealmeida commented on a diff in pull request #20862: fix: overwrite update override columns on PUT /dataset
Posted by GitBox <gi...@apache.org>.
betodealmeida commented on code in PR #20862:
URL: https://github.com/apache/superset/pull/20862#discussion_r932580741
##########
tests/integration_tests/datasets/api_tests.py:
##########
@@ -777,6 +777,35 @@ def test_update_dataset_item_w_override_columns(self):
db.session.delete(dataset)
db.session.commit()
+ def test_update_dataset_item_w_override_columns_same_columns(self):
+ """
+ Dataset API: Test update dataset with override columns
+ """
+ if backend() == "sqlite":
+ return
+
+ # Add default dataset
+ main_db = get_main_database()
+ dataset = self.insert_dataset("flights", [self.get_user("admin").id], main_db)
+
+ self.login(username="admin")
+ dataset_data = {
+ "sql": "SELECT * from flights ",
+ "columns": [
+ {"column_name": "YEAR", "type": "INT", "is_dttm": False},
+ {"column_name": "MONTH", "type": "INT", "is_dttm": False},
+ {"column_name": "DAY", "type": "INT", "is_dttm": False},
+ ],
+ }
+ uri = f"api/v1/dataset/{dataset.id}?override_columns=true"
+ rv = self.put_assert_metric(uri, dataset_data, "put")
+ assert rv.status_code == 200
+
+ columns = db.session.query(TableColumn).filter_by(table_id=dataset.id).all()
+ assert len(columns) == 3
Review Comment:
Can you add an assert that the number of columns was not 3 before the update? (ie, in line ~790) Otherwise we don't know if the update actually did anything.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] eschutho commented on a diff in pull request #20862: fix: overwrite update override columns on PUT /dataset
Posted by GitBox <gi...@apache.org>.
eschutho commented on code in PR #20862:
URL: https://github.com/apache/superset/pull/20862#discussion_r932764925
##########
superset/datasets/dao.py:
##########
@@ -147,14 +147,22 @@ def validate_metrics_uniqueness(dataset_id: int, metrics_names: List[str]) -> bo
@classmethod
def update(
- cls, model: SqlaTable, properties: Dict[str, Any], commit: bool = True
+ cls,
+ model: SqlaTable,
+ properties: Dict[str, Any],
+ commit: bool = True,
) -> Optional[SqlaTable]:
"""
Updates a Dataset model on the metadata DB
"""
if "columns" in properties:
- cls.update_columns(model, properties.pop("columns"), commit=commit)
+ cls.update_columns(
+ model,
+ properties.pop("columns"),
+ commit=commit,
+ override_columns=bool(properties.get("override_columns")),
+ )
if "metrics" in properties:
cls.update_metrics(model, properties.pop("metrics"), commit=commit)
Review Comment:
for put /dataset, does override_columns literally mean just the columns? Are we putting or patching here? Does it include the dataset? Wouldn't we be also overriding the metrics?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org