You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "mistercrunch (via GitHub)" <gi...@apache.org> on 2024/09/26 01:02:58 UTC
[PR] feat: add and use UUIDMixin for most models [superset]
mistercrunch opened a new pull request, #30398:
URL: https://github.com/apache/superset/pull/30398
Proof of concept for adding uuid to most table. This is just a DRAFT for now but could be used to sprinkly uuids to all models
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat: add and use UUIDMixin for most models [superset]
Posted by "mistercrunch (via GitHub)" <gi...@apache.org>.
mistercrunch commented on PR #30398:
URL: https://github.com/apache/superset/pull/30398#issuecomment-2502345994
Setting this as ready for review as we may want to repeat this pattern, and having `UuidMixin` around for use is a win
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat: add and use UUIDMixin for most models [superset]
Posted by "michael-s-molina (via GitHub)" <gi...@apache.org>.
michael-s-molina commented on code in PR #30398:
URL: https://github.com/apache/superset/pull/30398#discussion_r1860483099
##########
superset/migrations/versions/2024-09-25_17-59_7b17aa722e30_uuidmixin.py:
##########
@@ -0,0 +1,47 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""UUIDMixin
+
+Revision ID: 7b17aa722e30
+Revises: 48cbb571fa3a
+Create Date: 2024-09-25 17:59:21.028426
+
+"""
+
+import sqlalchemy as sa
+import sqlalchemy_utils
+from alembic import op
+
+# revision identifiers, used by Alembic.
+revision = "7b17aa722e30"
+down_revision = "48cbb571fa3a"
+
+
+def upgrade():
+ op.add_column(
Review Comment:
@mistercrunch could you make sure the migration checks if the column already exists when upgrading and downgrading?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat: add and use UUIDMixin for most models [superset]
Posted by "michael-s-molina (via GitHub)" <gi...@apache.org>.
michael-s-molina commented on code in PR #30398:
URL: https://github.com/apache/superset/pull/30398#discussion_r1862201552
##########
superset/migrations/versions/2024-09-25_17-59_7b17aa722e30_uuidmixin.py:
##########
@@ -0,0 +1,47 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""UUIDMixin
+
+Revision ID: 7b17aa722e30
+Revises: 48cbb571fa3a
+Create Date: 2024-09-25 17:59:21.028426
+
+"""
+
+import sqlalchemy as sa
+import sqlalchemy_utils
+from alembic import op
+
+# revision identifiers, used by Alembic.
+revision = "7b17aa722e30"
+down_revision = "48cbb571fa3a"
+
+
+def upgrade():
+ op.add_column(
Review Comment:
> Maybe a more important thing I could spend a cycle or two on would be to clean the mismatches. When running alembic, it becomes evident that there's a lot of differences between what's declared in models and what's in the physical database, resulting on a vanilla alembic run generating lots of output.
> If I was to align 100% on alembic, we could have a CI script that prevents deriving away between what's declared and how it gets stamped in the database.
That's a good idea 👍🏼
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat: add and use UUIDMixin for most models [superset]
Posted by "michael-s-molina (via GitHub)" <gi...@apache.org>.
michael-s-molina commented on code in PR #30398:
URL: https://github.com/apache/superset/pull/30398#discussion_r1862197951
##########
superset/migrations/versions/2024-09-25_17-59_7b17aa722e30_uuidmixin.py:
##########
@@ -0,0 +1,47 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""UUIDMixin
+
+Revision ID: 7b17aa722e30
+Revises: 48cbb571fa3a
+Create Date: 2024-09-25 17:59:21.028426
+
+"""
+
+import sqlalchemy as sa
+import sqlalchemy_utils
+from alembic import op
+
+# revision identifiers, used by Alembic.
+revision = "7b17aa722e30"
+down_revision = "48cbb571fa3a"
+
+
+def upgrade():
+ op.add_column(
Review Comment:
> Not sure if false-negative or swallowed errors on db migrations is a thing, personally I don't think I've ever seen it.
I think we should not swallow the errors. It's more about making sure that migrations are more resilient. Dropping a column that does not exist or adding a column that matches the one in the database (same config) should not throw errors. We could log warnings instead so that admins are aware.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat: add and use UUIDMixin for most models [superset]
Posted by "mistercrunch (via GitHub)" <gi...@apache.org>.
mistercrunch commented on code in PR #30398:
URL: https://github.com/apache/superset/pull/30398#discussion_r1861238853
##########
superset/migrations/versions/2024-09-25_17-59_7b17aa722e30_uuidmixin.py:
##########
@@ -0,0 +1,47 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""UUIDMixin
+
+Revision ID: 7b17aa722e30
+Revises: 48cbb571fa3a
+Create Date: 2024-09-25 17:59:21.028426
+
+"""
+
+import sqlalchemy as sa
+import sqlalchemy_utils
+from alembic import op
+
+# revision identifiers, used by Alembic.
+revision = "7b17aa722e30"
+down_revision = "48cbb571fa3a"
+
+
+def upgrade():
+ op.add_column(
Review Comment:
Seem like a decent idea, though wondering how to best to this. Not sure if false-negative or swallowed errors on db migrations is a thing, personally I don't think I've ever seen it. I've seen migrations that fail half-way through and bad states, but always with a clear error message.
Taking a step back and thinking about improving migrations-related code / process. One thing I'd like seen is collapsing/cleaning database migrations.
```
$ find superset/migrations/versions/|wc -l
980
```
Currently we have 980 migrations, I could collapse since say 1.0.0 or 2.0.0 and end up with much less migrations. Not sure if worth our while.
Maybe a more important thing I could spend a cycle or two on would be to clean the mismatches. When running alembic, it becomes evident that there's a lot of differences between what's declared in models and what's in the physical database, resulting on a vanilla alembic run generating lots of output.
If I was to align 100% on alembic, we could have a CI script that prevents deriving away between what's declared and how it gets stamped in the database.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat: add and use UUIDMixin for most models [superset]
Posted by "mistercrunch (via GitHub)" <gi...@apache.org>.
mistercrunch merged PR #30398:
URL: https://github.com/apache/superset/pull/30398
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat: add and use UUIDMixin for most models [superset]
Posted by "michael-s-molina (via GitHub)" <gi...@apache.org>.
michael-s-molina commented on code in PR #30398:
URL: https://github.com/apache/superset/pull/30398#discussion_r1860483099
##########
superset/migrations/versions/2024-09-25_17-59_7b17aa722e30_uuidmixin.py:
##########
@@ -0,0 +1,47 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""UUIDMixin
+
+Revision ID: 7b17aa722e30
+Revises: 48cbb571fa3a
+Create Date: 2024-09-25 17:59:21.028426
+
+"""
+
+import sqlalchemy as sa
+import sqlalchemy_utils
+from alembic import op
+
+# revision identifiers, used by Alembic.
+revision = "7b17aa722e30"
+down_revision = "48cbb571fa3a"
+
+
+def upgrade():
+ op.add_column(
Review Comment:
@mistercrunch could you make sure the migration checks if the column exists when upgrading and downgrading?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat: add and use UUIDMixin for most models [superset]
Posted by "villebro (via GitHub)" <gi...@apache.org>.
villebro commented on code in PR #30398:
URL: https://github.com/apache/superset/pull/30398#discussion_r1777447705
##########
superset/models/core.py:
##########
@@ -102,7 +102,7 @@ class KeyValue(Model): # pylint: disable=too-few-public-methods
value = Column(utils.MediumText(), nullable=False)
-class CssTemplate(Model, AuditMixinNullable):
+class CssTemplate(Model, AuditMixinNullable, UUIDMixin):
Review Comment:
This was already incorrect, but in general it's idiomatic to place the mixin _before_ the main class to ensure correct MRO (otherwise functionality in the main class may overwrite functionality in the mixin):
```suggestion
class CssTemplate(AuditMixinNullable, UUIDMixin, Model):
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat: add and use UUIDMixin for most models [superset]
Posted by "villebro (via GitHub)" <gi...@apache.org>.
villebro commented on code in PR #30398:
URL: https://github.com/apache/superset/pull/30398#discussion_r1777447705
##########
superset/models/core.py:
##########
@@ -102,7 +102,7 @@ class KeyValue(Model): # pylint: disable=too-few-public-methods
value = Column(utils.MediumText(), nullable=False)
-class CssTemplate(Model, AuditMixinNullable):
+class CssTemplate(Model, AuditMixinNullable, UUIDMixin):
Review Comment:
This was already incorrect, but in general it's idiomatic to place the mixin _before_ the main class to ensure MRO (otherwise functionality in the main class will overwrite functionality in the mixin):
```suggestion
class CssTemplate(AuditMixinNullable, UUIDMixin, Model):
```
##########
superset/models/helpers.py:
##########
@@ -167,11 +167,17 @@ def convert_uuids(obj: Any) -> Any:
return obj
-class ImportExportMixin:
+class UUIDMixin: # pylint: disable=too-few-public-methods
Review Comment:
Nice clarification!
##########
superset/models/core.py:
##########
@@ -1174,7 +1174,7 @@ class FavStarClassName(StrEnum):
DASHBOARD = "Dashboard"
-class FavStar(Model): # pylint: disable=too-few-public-methods
+class FavStar(Model, UUIDMixin):
Review Comment:
Same here:
```suggestion
class FavStar(UUIDMixin, Model):
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat: add and use UUIDMixin for most models [superset]
Posted by "villebro (via GitHub)" <gi...@apache.org>.
villebro commented on code in PR #30398:
URL: https://github.com/apache/superset/pull/30398#discussion_r1777447705
##########
superset/models/core.py:
##########
@@ -102,7 +102,7 @@ class KeyValue(Model): # pylint: disable=too-few-public-methods
value = Column(utils.MediumText(), nullable=False)
-class CssTemplate(Model, AuditMixinNullable):
+class CssTemplate(Model, AuditMixinNullable, UUIDMixin):
Review Comment:
This was already incorrect, but in general it's idiomatic to place the mixin _before_ the main class to ensure MRO (otherwise functionality in the main class may overwrite functionality in the mixin):
```suggestion
class CssTemplate(AuditMixinNullable, UUIDMixin, Model):
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat: add and use UUIDMixin for most models [superset]
Posted by "michael-s-molina (via GitHub)" <gi...@apache.org>.
michael-s-molina commented on code in PR #30398:
URL: https://github.com/apache/superset/pull/30398#discussion_r1860484136
##########
superset/migrations/versions/2024-09-25_17-59_7b17aa722e30_uuidmixin.py:
##########
@@ -0,0 +1,47 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""UUIDMixin
+
+Revision ID: 7b17aa722e30
+Revises: 48cbb571fa3a
+Create Date: 2024-09-25 17:59:21.028426
+
+"""
+
+import sqlalchemy as sa
+import sqlalchemy_utils
+from alembic import op
+
+# revision identifiers, used by Alembic.
+revision = "7b17aa722e30"
+down_revision = "48cbb571fa3a"
+
+
+def upgrade():
+ op.add_column(
Review Comment:
I think it would be good to create a utility for adding/removing columns that always performs this check.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org