You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by GitBox <gi...@apache.org> on 2022/11/15 18:24:39 UTC
[GitHub] [superset] eschutho opened a new pull request, #22129: add back database lookup from sip 68 revert
eschutho opened a new pull request, #22129:
URL: https://github.com/apache/superset/pull/22129
<!---
Please write the PR title following the conventions at https://www.conventionalcommits.org/en/v1.0.0/
Example:
fix(dashboard): load charts correctly
-->
### SUMMARY
<!--- Describe the change below, including rationale and design decisions -->
### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
<!--- Skip this if not applicable -->
### TESTING INSTRUCTIONS
<!--- Required! What steps can be taken to manually verify the changes? -->
### ADDITIONAL INFORMATION
<!--- Check any relevant boxes with "x" -->
<!--- HINT: Include "Fixes #nnn" if you are fixing an existing issue -->
- [ ] Has associated issue:
- [ ] Required feature flags:
- [ ] Changes UI
- [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351))
- [ ] Migration is atomic, supports rollback & is backwards-compatible
- [ ] Confirm DB migration upgrade and downgrade tested
- [ ] Runtime estimates and downtime expectations provided
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] eschutho commented on a diff in pull request #22129: fix: add back database lookup from sip 68 revert
Posted by GitBox <gi...@apache.org>.
eschutho commented on code in PR #22129:
URL: https://github.com/apache/superset/pull/22129#discussion_r1023152101
##########
superset/connectors/sqla/models.py:
##########
@@ -2117,11 +2137,53 @@ def after_update(
sqla_table: "SqlaTable",
) -> None:
"""
- Update dataset permissions after update
+ Update dataset permissions
"""
# set permissions
security_manager.dataset_after_update(mapper, connection, sqla_table)
+ # TODO: the shadow writing is deprecated
+ inspector = inspect(sqla_table)
+ session = inspector.session
+
+ # double-check that ``UPDATE``s are actually pending (this method is called even
+ # for instances that have no net changes to their column-based attributes)
+ if not session.is_modified(sqla_table, include_collections=True):
+ return
+
+ # find the dataset from the known instance list first
+ # (it could be either from a previous query or newly created)
+ dataset = next(
+ find_cached_objects_in_session(
+ session, NewDataset, uuids=[sqla_table.uuid]
+ ),
+ None,
+ )
+ # if not found, pull from database
+ if not dataset:
+ dataset = (
+ session.query(NewDataset).filter_by(uuid=sqla_table.uuid).one_or_none()
+ )
+ if not dataset:
+ sqla_table.write_shadow_dataset()
+ return
+
+ def write_shadow_dataset(
+ self: "SqlaTable",
+ ) -> None:
+ """
+ This method is deprecated
+ """
+ session = inspect(self).session
+ # most of the write_shadow_dataset functionality has been removed
+ # but leaving this portion in
+ # to remove later because it is adding a Database relationship to the session
+ # and there is some functionality that depends on this
+ if self.database_id and (
+ not self.database or self.database.id != self.database_id
+ ):
+ self.database = session.query(Database).filter_by(id=self.database_id).one()
Review Comment:
We're adding the database to the model here during flush, so removing this is what caused some issues for existing code that relied on this side effect.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] eschutho commented on a diff in pull request #22129: add back database lookup from sip 68 revert
Posted by GitBox <gi...@apache.org>.
eschutho commented on code in PR #22129:
URL: https://github.com/apache/superset/pull/22129#discussion_r1023131839
##########
superset/datasets/dao.py:
##########
@@ -37,26 +36,6 @@ class DatasetDAO(BaseDAO): # pylint: disable=too-many-public-methods
model_cls = SqlaTable
base_filter = DatasourceFilter
- @classmethod
- def find_by_ids(cls, model_ids: Union[List[str], List[int]]) -> List[SqlaTable]:
- """
- Find a List of models by a list of ids, if defined applies `base_filter`
- """
- id_col = getattr(SqlaTable, cls.id_column_name, None)
- if id_col is None:
- return []
-
- # the joinedload option ensures that the database is
- # available in the session later and not lazy loaded
- query = (
- db.session.query(SqlaTable)
- .options(joinedload(SqlaTable.database))
- .filter(id_col.in_(model_ids))
- )
- data_model = SQLAInterface(SqlaTable, db.session)
- query = DatasourceFilter(cls.id_column_name, data_model).apply(query, None)
- return query.all()
-
Review Comment:
This method was added in to the original PR to eager load the database.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] codecov[bot] commented on pull request #22129: add back database lookup from sip 68 revert
Posted by GitBox <gi...@apache.org>.
codecov[bot] commented on PR #22129:
URL: https://github.com/apache/superset/pull/22129#issuecomment-1315711174
# [Codecov](https://codecov.io/gh/apache/superset/pull/22129?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#22129](https://codecov.io/gh/apache/superset/pull/22129?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (10b1bd1) into [master](https://codecov.io/gh/apache/superset/commit/06f87e1467df0a306ef7ae0349b8494cd844a730?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (06f87e1) will **decrease** coverage by `11.45%`.
> The diff coverage is `84.61%`.
```diff
@@ Coverage Diff @@
## master #22129 +/- ##
===========================================
- Coverage 66.97% 55.52% -11.46%
===========================================
Files 1832 1832
Lines 69905 69918 +13
Branches 7570 7570
===========================================
- Hits 46820 38820 -8000
- Misses 21127 29140 +8013
Partials 1958 1958
```
| Flag | Coverage Δ | |
|---|---|---|
| hive | `?` | |
| mysql | `?` | |
| postgres | `?` | |
| presto | `52.50% <80.76%> (+0.03%)` | :arrow_up: |
| python | `57.42% <84.61%> (-23.95%)` | :arrow_down: |
| sqlite | `?` | |
| unit | `50.86% <80.76%> (+0.04%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/superset/pull/22129?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [superset/connectors/sqla/models.py](https://codecov.io/gh/apache/superset/pull/22129/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvY29ubmVjdG9ycy9zcWxhL21vZGVscy5weQ==) | `74.80% <84.00%> (-15.42%)` | :arrow_down: |
| [superset/datasets/dao.py](https://codecov.io/gh/apache/superset/pull/22129/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGF0YXNldHMvZGFvLnB5) | `44.59% <100.00%> (-49.71%)` | :arrow_down: |
| [superset/utils/dashboard\_import\_export.py](https://codecov.io/gh/apache/superset/pull/22129/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdXRpbHMvZGFzaGJvYXJkX2ltcG9ydF9leHBvcnQucHk=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [superset/tags/core.py](https://codecov.io/gh/apache/superset/pull/22129/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdGFncy9jb3JlLnB5) | `4.54% <0.00%> (-95.46%)` | :arrow_down: |
| [superset/key\_value/commands/update.py](https://codecov.io/gh/apache/superset/pull/22129/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL3VwZGF0ZS5weQ==) | `0.00% <0.00%> (-90.91%)` | :arrow_down: |
| [superset/key\_value/commands/delete.py](https://codecov.io/gh/apache/superset/pull/22129/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL2RlbGV0ZS5weQ==) | `0.00% <0.00%> (-87.88%)` | :arrow_down: |
| [superset/db\_engines/hive.py](https://codecov.io/gh/apache/superset/pull/22129/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGJfZW5naW5lcy9oaXZlLnB5) | `0.00% <0.00%> (-85.19%)` | :arrow_down: |
| [superset/key\_value/commands/delete\_expired.py](https://codecov.io/gh/apache/superset/pull/22129/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL2RlbGV0ZV9leHBpcmVkLnB5) | `0.00% <0.00%> (-84.00%)` | :arrow_down: |
| [superset/dashboards/commands/importers/v0.py](https://codecov.io/gh/apache/superset/pull/22129/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGFzaGJvYXJkcy9jb21tYW5kcy9pbXBvcnRlcnMvdjAucHk=) | `15.62% <0.00%> (-76.25%)` | :arrow_down: |
| [superset/datasets/commands/create.py](https://codecov.io/gh/apache/superset/pull/22129/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGF0YXNldHMvY29tbWFuZHMvY3JlYXRlLnB5) | `30.61% <0.00%> (-69.39%)` | :arrow_down: |
| ... and [285 more](https://codecov.io/gh/apache/superset/pull/22129/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] eschutho merged pull request #22129: fix: add back database lookup from sip 68 revert
Posted by GitBox <gi...@apache.org>.
eschutho merged PR #22129:
URL: https://github.com/apache/superset/pull/22129
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org