[GitHub] [incubator-superset] PU-101 commented on a change in pull request #10244: fix: error occurred when fetching large amount of columns

[GitBox <gi...@apache.org>]

PU-101 commented on a change in pull request #10244:
URL: https://github.com/apache/incubator-superset/pull/10244#discussion_r451354211



##########
File path: superset/connectors/sqla/models.py
##########
@@ -1188,12 +1188,16 @@ def fetch_metadata(self, commit: bool = True) -> None:
         any_date_col = None
         db_engine_spec = self.database.db_engine_spec
         db_dialect = self.database.get_dialect()
-        dbcols = (
-            db.session.query(TableColumn)
-            .filter(TableColumn.table == self)
-            .filter(or_(TableColumn.column_name == col.name for col in table_.columns))
-        )
-        dbcols = {dbcol.column_name: dbcol for dbcol in dbcols}
+        
+        try:
+            dbcols = (
+                db.session.query(TableColumn)
+                .filter(TableColumn.table == self)
+                .filter(TableColumn.column_name.in_([col.name for col in table.columns]))
+            )
+            dbcols = {dbcol.column_name: dbcol for dbcol in dbcols}
+        except Exception as e:
+            logger.error("Failed to get columns: " + str(e))

Review comment:
       we are trying to catch any exception when we query `table_columns` in sqlite3. If there is no 'try...except...' block, the table seems to be added normally except columns，but there's no any error log. so, i think it's necessary to try to catch any exception here.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org