You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "Antonio-RiveroMartnez (via GitHub)" <gi...@apache.org> on 2023/02/24 14:46:40 UTC
[GitHub] [superset] Antonio-RiveroMartnez opened a new pull request, #23191: fix(dataset): Respect columns configuration when duplicating datasets
Antonio-RiveroMartnez opened a new pull request, #23191:
URL: https://github.com/apache/superset/pull/23191
### SUMMARY
When duplicating datasets, we currently are forcing columns on the new dataset to have `groupby` and `filreable` set to `True` even if the user marked those as false in the base dataset. With this changes we are fixing that. Also did a small refactor in how we raise exceptions so they are all in the `validate` method and using existing exceptions.
### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
Before:

After:

### TESTING INSTRUCTIONS
1. Create a Virtual dataset
2. Create a `calculated column` in your new dataset
3. Check `isFilterable` for the new column and save the changes
4. Open the new dataset for edition and uncheck `isFilterable` so it is False
5. Duplicate the dataset
6. The new dataset must have `isFilterable` unchecked
### ADDITIONAL INFORMATION
<!--- Check any relevant boxes with "x" -->
<!--- HINT: Include "Fixes #nnn" if you are fixing an existing issue -->
- [ ] Has associated issue:
- [ ] Required feature flags:
- [ ] Changes UI
- [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351))
- [ ] Migration is atomic, supports rollback & is backwards-compatible
- [ ] Confirm DB migration upgrade and downgrade tested
- [ ] Runtime estimates and downtime expectations provided
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(dataset): Respect columns configuration when duplicating datasets [superset]
Posted by "Antonio-RiveroMartnez (via GitHub)" <gi...@apache.org>.
Antonio-RiveroMartnez closed pull request #23191: fix(dataset): Respect columns configuration when duplicating datasets
URL: https://github.com/apache/superset/pull/23191
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] Antonio-RiveroMartnez commented on a diff in pull request #23191: fix(dataset): Respect columns configuration when duplicating datasets
Posted by "Antonio-RiveroMartnez (via GitHub)" <gi...@apache.org>.
Antonio-RiveroMartnez commented on code in PR #23191:
URL: https://github.com/apache/superset/pull/23191#discussion_r1123518573
##########
superset/datasets/commands/duplicate.py:
##########
@@ -44,49 +43,39 @@
class DuplicateDatasetCommand(CreateMixin, BaseCommand):
def __init__(self, data: Dict[str, Any]) -> None:
- self._base_model: SqlaTable = SqlaTable()
+ self._base_model: Optional[SqlaTable] = None
self._properties = data.copy()
+ self._database: Optional[Database] = None
def run(self) -> Model:
self.validate()
try:
- database_id = self._base_model.database_id
table_name = self._properties["table_name"]
owners = self._properties["owners"]
- database = db.session.query(Database).get(database_id)
- if not database:
- raise SupersetErrorException(
- SupersetError(
- message=__("The database was not found."),
- error_type=SupersetErrorType.DATABASE_NOT_FOUND_ERROR,
- level=ErrorLevel.ERROR,
- ),
- status=404,
- )
table = SqlaTable(table_name=table_name, owners=owners)
- table.database = database
- table.schema = self._base_model.schema
- table.template_params = self._base_model.template_params
+ table.database = self._database # type: ignore
Review Comment:
Addressing review comments, we are now leaning for readability.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] eschutho commented on a diff in pull request #23191: fix(dataset): Respect columns configuration when duplicating datasets
Posted by "eschutho (via GitHub)" <gi...@apache.org>.
eschutho commented on code in PR #23191:
URL: https://github.com/apache/superset/pull/23191#discussion_r1117480001
##########
superset/datasets/commands/duplicate.py:
##########
@@ -44,60 +43,51 @@
class DuplicateDatasetCommand(CreateMixin, BaseCommand):
def __init__(self, data: Dict[str, Any]) -> None:
- self._base_model: SqlaTable = SqlaTable()
+ self._base_model: Optional[SqlaTable] = None
self._properties = data.copy()
+ self._database: Optional[Database] = None
def run(self) -> Model:
self.validate()
try:
- database_id = self._base_model.database_id
- table_name = self._properties["table_name"]
- owners = self._properties["owners"]
- database = db.session.query(Database).get(database_id)
- if not database:
- raise SupersetErrorException(
- SupersetError(
- message=__("The database was not found."),
- error_type=SupersetErrorType.DATABASE_NOT_FOUND_ERROR,
- level=ErrorLevel.ERROR,
- ),
- status=404,
- )
- table = SqlaTable(table_name=table_name, owners=owners)
- table.database = database
- table.schema = self._base_model.schema
- table.template_params = self._base_model.template_params
- table.is_sqllab_view = True
- table.sql = ParsedQuery(self._base_model.sql).stripped()
- db.session.add(table)
- cols = []
- for config_ in self._base_model.columns:
- column_name = config_.column_name
- col = TableColumn(
- column_name=column_name,
- verbose_name=config_.verbose_name,
- expression=config_.expression,
- filterable=True,
- groupby=True,
- is_dttm=config_.is_dttm,
- type=config_.type,
- description=config_.description,
- )
- cols.append(col)
- table.columns = cols
- mets = []
- for config_ in self._base_model.metrics:
- metric_name = config_.metric_name
- met = SqlMetric(
- metric_name=metric_name,
- verbose_name=config_.verbose_name,
- expression=config_.expression,
- metric_type=config_.metric_type,
- description=config_.description,
- )
- mets.append(met)
- table.metrics = mets
- db.session.commit()
+ if self._base_model and self._database:
Review Comment:
@betodealmeida or @hughhhh what do you think here? The validate method will raise if self._base_model or self._database are falsy. Technically those values couldn't be false on line 53. I'm leaning for readability sake to just skip the pypy warning so that the reader understands that those values couldn't be falsy here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(dataset): Respect columns configuration when duplicating datasets [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on PR #23191:
URL: https://github.com/apache/superset/pull/23191#issuecomment-1960476958
@Antonio-RiveroMartnez do we still intend to get this through, or should we close it out?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] eschutho commented on a diff in pull request #23191: fix(dataset): Respect columns configuration when duplicating datasets
Posted by "eschutho (via GitHub)" <gi...@apache.org>.
eschutho commented on code in PR #23191:
URL: https://github.com/apache/superset/pull/23191#discussion_r1139297387
##########
superset/datasets/commands/duplicate.py:
##########
@@ -44,49 +43,39 @@
class DuplicateDatasetCommand(CreateMixin, BaseCommand):
def __init__(self, data: Dict[str, Any]) -> None:
- self._base_model: SqlaTable = SqlaTable()
+ self._base_model: Optional[SqlaTable] = None
self._properties = data.copy()
+ self._database: Optional[Database] = None
def run(self) -> Model:
self.validate()
try:
- database_id = self._base_model.database_id
table_name = self._properties["table_name"]
owners = self._properties["owners"]
- database = db.session.query(Database).get(database_id)
- if not database:
- raise SupersetErrorException(
- SupersetError(
- message=__("The database was not found."),
- error_type=SupersetErrorType.DATABASE_NOT_FOUND_ERROR,
- level=ErrorLevel.ERROR,
- ),
- status=404,
- )
table = SqlaTable(table_name=table_name, owners=owners)
- table.database = database
- table.schema = self._base_model.schema
- table.template_params = self._base_model.template_params
+ table.database = self._database # type: ignore
Review Comment:
@Antonio-RiveroMartnez thanks for this! I've asked @betodealmeida to see if he can suggest any other patterns here, since now based on my feedback there are a lot of `ignore`s. We talked about validating sooner, or moving some of the validation logic into the init. If there are any suggestions that you have to make this cleaner, we can collectively come up with a new pattern.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] Antonio-RiveroMartnez commented on a diff in pull request #23191: fix(dataset): Respect columns configuration when duplicating datasets
Posted by "Antonio-RiveroMartnez (via GitHub)" <gi...@apache.org>.
Antonio-RiveroMartnez commented on code in PR #23191:
URL: https://github.com/apache/superset/pull/23191#discussion_r1117123483
##########
superset/datasets/commands/duplicate.py:
##########
@@ -44,60 +43,51 @@
class DuplicateDatasetCommand(CreateMixin, BaseCommand):
def __init__(self, data: Dict[str, Any]) -> None:
- self._base_model: SqlaTable = SqlaTable()
+ self._base_model: Optional[SqlaTable] = None
self._properties = data.copy()
+ self._database: Optional[Database] = None
def run(self) -> Model:
self.validate()
try:
- database_id = self._base_model.database_id
- table_name = self._properties["table_name"]
- owners = self._properties["owners"]
- database = db.session.query(Database).get(database_id)
- if not database:
- raise SupersetErrorException(
- SupersetError(
- message=__("The database was not found."),
- error_type=SupersetErrorType.DATABASE_NOT_FOUND_ERROR,
- level=ErrorLevel.ERROR,
- ),
- status=404,
- )
- table = SqlaTable(table_name=table_name, owners=owners)
- table.database = database
- table.schema = self._base_model.schema
- table.template_params = self._base_model.template_params
- table.is_sqllab_view = True
- table.sql = ParsedQuery(self._base_model.sql).stripped()
- db.session.add(table)
- cols = []
- for config_ in self._base_model.columns:
- column_name = config_.column_name
- col = TableColumn(
- column_name=column_name,
- verbose_name=config_.verbose_name,
- expression=config_.expression,
- filterable=True,
- groupby=True,
- is_dttm=config_.is_dttm,
- type=config_.type,
- description=config_.description,
- )
- cols.append(col)
- table.columns = cols
- mets = []
- for config_ in self._base_model.metrics:
- metric_name = config_.metric_name
- met = SqlMetric(
- metric_name=metric_name,
- verbose_name=config_.verbose_name,
- expression=config_.expression,
- metric_type=config_.metric_type,
- description=config_.description,
- )
- mets.append(met)
- table.metrics = mets
- db.session.commit()
+ if self._base_model and self._database:
Review Comment:
Forced to use a check before so `mypy` doesn't complain when assigning the properties to the new `dataset` now that we use `Optional`. The majority of the code inside this block remains untouched.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] Antonio-RiveroMartnez commented on a diff in pull request #23191: fix(dataset): Respect columns configuration when duplicating datasets
Posted by "Antonio-RiveroMartnez (via GitHub)" <gi...@apache.org>.
Antonio-RiveroMartnez commented on code in PR #23191:
URL: https://github.com/apache/superset/pull/23191#discussion_r1123605004
##########
superset/datasets/commands/duplicate.py:
##########
@@ -44,60 +43,51 @@
class DuplicateDatasetCommand(CreateMixin, BaseCommand):
def __init__(self, data: Dict[str, Any]) -> None:
- self._base_model: SqlaTable = SqlaTable()
+ self._base_model: Optional[SqlaTable] = None
self._properties = data.copy()
+ self._database: Optional[Database] = None
def run(self) -> Model:
self.validate()
try:
- database_id = self._base_model.database_id
- table_name = self._properties["table_name"]
- owners = self._properties["owners"]
- database = db.session.query(Database).get(database_id)
- if not database:
- raise SupersetErrorException(
- SupersetError(
- message=__("The database was not found."),
- error_type=SupersetErrorType.DATABASE_NOT_FOUND_ERROR,
- level=ErrorLevel.ERROR,
- ),
- status=404,
- )
- table = SqlaTable(table_name=table_name, owners=owners)
- table.database = database
- table.schema = self._base_model.schema
- table.template_params = self._base_model.template_params
- table.is_sqllab_view = True
- table.sql = ParsedQuery(self._base_model.sql).stripped()
- db.session.add(table)
- cols = []
- for config_ in self._base_model.columns:
- column_name = config_.column_name
- col = TableColumn(
- column_name=column_name,
- verbose_name=config_.verbose_name,
- expression=config_.expression,
- filterable=True,
- groupby=True,
- is_dttm=config_.is_dttm,
- type=config_.type,
- description=config_.description,
- )
- cols.append(col)
- table.columns = cols
- mets = []
- for config_ in self._base_model.metrics:
- metric_name = config_.metric_name
- met = SqlMetric(
- metric_name=metric_name,
- verbose_name=config_.verbose_name,
- expression=config_.expression,
- metric_type=config_.metric_type,
- description=config_.description,
- )
- mets.append(met)
- table.metrics = mets
- db.session.commit()
+ if self._base_model and self._database:
Review Comment:
Hey @eschutho I changed it now, based on your comments. Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] codecov[bot] commented on pull request #23191: fix(dataset): Respect columns configuration when duplicating datasets
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #23191:
URL: https://github.com/apache/superset/pull/23191#issuecomment-1443797769
# [Codecov](https://codecov.io/gh/apache/superset/pull/23191?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#23191](https://codecov.io/gh/apache/superset/pull/23191?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (e0f0d9e) into [master](https://codecov.io/gh/apache/superset/commit/d0c54cddb074887bebc9168ddcdca3e6ab898279?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (d0c54cd) will **decrease** coverage by `11.26%`.
> The diff coverage is `8.82%`.
> :exclamation: Current head e0f0d9e differs from pull request most recent head 31df454. Consider uploading reports for the commit 31df454 to get more accurate results
```diff
@@ Coverage Diff @@
## master #23191 +/- ##
===========================================
- Coverage 67.46% 56.21% -11.26%
===========================================
Files 1899 1899
Lines 73100 73099 -1
Branches 7899 7899
===========================================
- Hits 49320 41089 -8231
- Misses 21755 29985 +8230
Partials 2025 2025
```
| Flag | Coverage Δ | |
|---|---|---|
| hive | `?` | |
| mysql | `?` | |
| postgres | `?` | |
| presto | `52.75% <8.82%> (-0.01%)` | :arrow_down: |
| python | `58.93% <8.82%> (-23.28%)` | :arrow_down: |
| unit | `52.59% <8.82%> (-0.01%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/superset/pull/23191?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [superset/datasets/commands/duplicate.py](https://codecov.io/gh/apache/superset/pull/23191?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGF0YXNldHMvY29tbWFuZHMvZHVwbGljYXRlLnB5) | `27.63% <8.82%> (-63.28%)` | :arrow_down: |
| [superset/utils/dashboard\_import\_export.py](https://codecov.io/gh/apache/superset/pull/23191?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdXRpbHMvZGFzaGJvYXJkX2ltcG9ydF9leHBvcnQucHk=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [superset/tags/core.py](https://codecov.io/gh/apache/superset/pull/23191?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdGFncy9jb3JlLnB5) | `4.54% <0.00%> (-95.46%)` | :arrow_down: |
| [superset/key\_value/commands/update.py](https://codecov.io/gh/apache/superset/pull/23191?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL3VwZGF0ZS5weQ==) | `0.00% <0.00%> (-90.91%)` | :arrow_down: |
| [superset/key\_value/commands/delete.py](https://codecov.io/gh/apache/superset/pull/23191?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL2RlbGV0ZS5weQ==) | `0.00% <0.00%> (-87.88%)` | :arrow_down: |
| [superset/key\_value/commands/delete\_expired.py](https://codecov.io/gh/apache/superset/pull/23191?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL2RlbGV0ZV9leHBpcmVkLnB5) | `0.00% <0.00%> (-84.00%)` | :arrow_down: |
| [superset/dashboards/commands/importers/v0.py](https://codecov.io/gh/apache/superset/pull/23191?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGFzaGJvYXJkcy9jb21tYW5kcy9pbXBvcnRlcnMvdjAucHk=) | `15.62% <0.00%> (-76.25%)` | :arrow_down: |
| [superset/datasets/commands/create.py](https://codecov.io/gh/apache/superset/pull/23191?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGF0YXNldHMvY29tbWFuZHMvY3JlYXRlLnB5) | `30.61% <0.00%> (-69.39%)` | :arrow_down: |
| [superset/datasets/commands/update.py](https://codecov.io/gh/apache/superset/pull/23191?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGF0YXNldHMvY29tbWFuZHMvdXBkYXRlLnB5) | `25.00% <0.00%> (-69.05%)` | :arrow_down: |
| [superset/datasets/commands/importers/v0.py](https://codecov.io/gh/apache/superset/pull/23191?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGF0YXNldHMvY29tbWFuZHMvaW1wb3J0ZXJzL3YwLnB5) | `24.03% <0.00%> (-69.00%)` | :arrow_down: |
| ... and [291 more](https://codecov.io/gh/apache/superset/pull/23191?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(dataset): Respect columns configuration when duplicating datasets [superset]
Posted by "Antonio-RiveroMartnez (via GitHub)" <gi...@apache.org>.
Antonio-RiveroMartnez commented on PR #23191:
URL: https://github.com/apache/superset/pull/23191#issuecomment-1960497639
Im'm closing this out @evans , thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] eschutho commented on a diff in pull request #23191: fix(dataset): Respect columns configuration when duplicating datasets
Posted by "eschutho (via GitHub)" <gi...@apache.org>.
eschutho commented on code in PR #23191:
URL: https://github.com/apache/superset/pull/23191#discussion_r1139297387
##########
superset/datasets/commands/duplicate.py:
##########
@@ -44,49 +43,39 @@
class DuplicateDatasetCommand(CreateMixin, BaseCommand):
def __init__(self, data: Dict[str, Any]) -> None:
- self._base_model: SqlaTable = SqlaTable()
+ self._base_model: Optional[SqlaTable] = None
self._properties = data.copy()
+ self._database: Optional[Database] = None
def run(self) -> Model:
self.validate()
try:
- database_id = self._base_model.database_id
table_name = self._properties["table_name"]
owners = self._properties["owners"]
- database = db.session.query(Database).get(database_id)
- if not database:
- raise SupersetErrorException(
- SupersetError(
- message=__("The database was not found."),
- error_type=SupersetErrorType.DATABASE_NOT_FOUND_ERROR,
- level=ErrorLevel.ERROR,
- ),
- status=404,
- )
table = SqlaTable(table_name=table_name, owners=owners)
- table.database = database
- table.schema = self._base_model.schema
- table.template_params = self._base_model.template_params
+ table.database = self._database # type: ignore
Review Comment:
@Antonio-RiveroMartnez thanks for this! I've asked @betodealmeida to see if he can suggest any other patterns here, since now based on my feedback there are a lot of `ignore`s. We talked about validating sooner, or moving some of the validation logic into the init. If there are any suggestions that you have to make this cleaner, we can propose a new pattern here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] Antonio-RiveroMartnez commented on a diff in pull request #23191: fix(dataset): Respect columns configuration when duplicating datasets
Posted by "Antonio-RiveroMartnez (via GitHub)" <gi...@apache.org>.
Antonio-RiveroMartnez commented on code in PR #23191:
URL: https://github.com/apache/superset/pull/23191#discussion_r1165766333
##########
superset/datasets/commands/duplicate.py:
##########
@@ -44,27 +43,19 @@
class DuplicateDatasetCommand(CreateMixin, BaseCommand):
def __init__(self, data: Dict[str, Any]) -> None:
- self._base_model: SqlaTable = SqlaTable()
+ self._base_model: Optional[SqlaTable] = None
self._properties = data.copy()
+ self._database: Optional[Database] = None
def run(self) -> Model:
self.validate()
+ assert self._database is not None
Review Comment:
Trying to be explicit about the fact that both (self._database and self._base_model) are not None moving forward. Also to help us removing the bunch of type check ignores we had before.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] Antonio-RiveroMartnez commented on a diff in pull request #23191: fix(dataset): Respect columns configuration when duplicating datasets
Posted by "Antonio-RiveroMartnez (via GitHub)" <gi...@apache.org>.
Antonio-RiveroMartnez commented on code in PR #23191:
URL: https://github.com/apache/superset/pull/23191#discussion_r1117120983
##########
superset/datasets/commands/duplicate.py:
##########
@@ -44,60 +43,51 @@
class DuplicateDatasetCommand(CreateMixin, BaseCommand):
def __init__(self, data: Dict[str, Any]) -> None:
- self._base_model: SqlaTable = SqlaTable()
+ self._base_model: Optional[SqlaTable] = None
Review Comment:
Changed to `Optional` since `find_by_id` could return `None`, and also it's more consistent how we handle such cases.
##########
superset/datasets/commands/duplicate.py:
##########
@@ -44,60 +43,51 @@
class DuplicateDatasetCommand(CreateMixin, BaseCommand):
def __init__(self, data: Dict[str, Any]) -> None:
- self._base_model: SqlaTable = SqlaTable()
+ self._base_model: Optional[SqlaTable] = None
self._properties = data.copy()
+ self._database: Optional[Database] = None
def run(self) -> Model:
self.validate()
try:
- database_id = self._base_model.database_id
- table_name = self._properties["table_name"]
- owners = self._properties["owners"]
- database = db.session.query(Database).get(database_id)
- if not database:
- raise SupersetErrorException(
- SupersetError(
- message=__("The database was not found."),
- error_type=SupersetErrorType.DATABASE_NOT_FOUND_ERROR,
- level=ErrorLevel.ERROR,
- ),
- status=404,
- )
- table = SqlaTable(table_name=table_name, owners=owners)
- table.database = database
- table.schema = self._base_model.schema
- table.template_params = self._base_model.template_params
- table.is_sqllab_view = True
- table.sql = ParsedQuery(self._base_model.sql).stripped()
- db.session.add(table)
- cols = []
- for config_ in self._base_model.columns:
- column_name = config_.column_name
- col = TableColumn(
- column_name=column_name,
- verbose_name=config_.verbose_name,
- expression=config_.expression,
- filterable=True,
- groupby=True,
- is_dttm=config_.is_dttm,
- type=config_.type,
- description=config_.description,
- )
- cols.append(col)
- table.columns = cols
- mets = []
- for config_ in self._base_model.metrics:
- metric_name = config_.metric_name
- met = SqlMetric(
- metric_name=metric_name,
- verbose_name=config_.verbose_name,
- expression=config_.expression,
- metric_type=config_.metric_type,
- description=config_.description,
- )
- mets.append(met)
- table.metrics = mets
- db.session.commit()
+ if self._base_model and self._database:
Review Comment:
Forced to use a check before so `pre-commit` doesn't complain when assigning the properties to the new `dataset`. The majority of the code inside this block remains untouched.
##########
superset/datasets/commands/duplicate.py:
##########
@@ -44,60 +43,51 @@
class DuplicateDatasetCommand(CreateMixin, BaseCommand):
def __init__(self, data: Dict[str, Any]) -> None:
- self._base_model: SqlaTable = SqlaTable()
+ self._base_model: Optional[SqlaTable] = None
self._properties = data.copy()
+ self._database: Optional[Database] = None
def run(self) -> Model:
self.validate()
try:
- database_id = self._base_model.database_id
- table_name = self._properties["table_name"]
- owners = self._properties["owners"]
- database = db.session.query(Database).get(database_id)
- if not database:
- raise SupersetErrorException(
Review Comment:
Changed to use `DatabaseNotFoundError` instead of constructing this here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] Antonio-RiveroMartnez commented on a diff in pull request #23191: fix(dataset): Respect columns configuration when duplicating datasets
Posted by "Antonio-RiveroMartnez (via GitHub)" <gi...@apache.org>.
Antonio-RiveroMartnez commented on code in PR #23191:
URL: https://github.com/apache/superset/pull/23191#discussion_r1117123483
##########
superset/datasets/commands/duplicate.py:
##########
@@ -44,60 +43,51 @@
class DuplicateDatasetCommand(CreateMixin, BaseCommand):
def __init__(self, data: Dict[str, Any]) -> None:
- self._base_model: SqlaTable = SqlaTable()
+ self._base_model: Optional[SqlaTable] = None
self._properties = data.copy()
+ self._database: Optional[Database] = None
def run(self) -> Model:
self.validate()
try:
- database_id = self._base_model.database_id
- table_name = self._properties["table_name"]
- owners = self._properties["owners"]
- database = db.session.query(Database).get(database_id)
- if not database:
- raise SupersetErrorException(
- SupersetError(
- message=__("The database was not found."),
- error_type=SupersetErrorType.DATABASE_NOT_FOUND_ERROR,
- level=ErrorLevel.ERROR,
- ),
- status=404,
- )
- table = SqlaTable(table_name=table_name, owners=owners)
- table.database = database
- table.schema = self._base_model.schema
- table.template_params = self._base_model.template_params
- table.is_sqllab_view = True
- table.sql = ParsedQuery(self._base_model.sql).stripped()
- db.session.add(table)
- cols = []
- for config_ in self._base_model.columns:
- column_name = config_.column_name
- col = TableColumn(
- column_name=column_name,
- verbose_name=config_.verbose_name,
- expression=config_.expression,
- filterable=True,
- groupby=True,
- is_dttm=config_.is_dttm,
- type=config_.type,
- description=config_.description,
- )
- cols.append(col)
- table.columns = cols
- mets = []
- for config_ in self._base_model.metrics:
- metric_name = config_.metric_name
- met = SqlMetric(
- metric_name=metric_name,
- verbose_name=config_.verbose_name,
- expression=config_.expression,
- metric_type=config_.metric_type,
- description=config_.description,
- )
- mets.append(met)
- table.metrics = mets
- db.session.commit()
+ if self._base_model and self._database:
Review Comment:
Forced to use a check before so `mypy` doesn't complain when assigning the properties to the new `dataset`. The majority of the code inside this block remains untouched.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org