[GitHub] [incubator-superset] dpgaspar commented on a change in pull request #11274: fix: error around latest partition in BigQuery

[GitBox <gi...@apache.org>]

dpgaspar commented on a change in pull request #11274:
URL: https://github.com/apache/incubator-superset/pull/11274#discussion_r508509779



##########
File path: superset/db_engine_specs/bigquery.py
##########
@@ -129,6 +129,25 @@ def _truncate_label(cls, label: str) -> str:
         """
         return "_" + hashlib.md5(label.encode("utf-8")).hexdigest()
 
+    @classmethod
+    def normalize_indexes(cls, indexes: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
+        """
+        Normalizes indexes for more consistency across db engines
+
+        :param indexes: Raw indexes as returned by SQLAlchemy
+        :return: cleaner, more aligned index definition
+        """
+        normalized_idxs = []
+        # Fixing a bug/behavior observed in pybigquery==0.4.15 where
+        # the index's `column_names` == [None]
+        # Here we're returning only non-None indexes
+        for ix in indexes:
+            column_names = ix.get("column_names") or []

Review comment:
       nit:
   ```
           for index in indexes:
               column_names = index.get("column_names", [])
   
   ```

##########
File path: tests/db_engine_specs/bigquery_tests.py
##########
@@ -122,6 +122,18 @@ def test_extra_table_metadata(self):
         )
         self.assertEqual(result, expected_result)
 
+    def test_normalize_indexes(self):
+        """
+        DB Eng Specs (bigquery): Test extra table metadata
+        """
+        indexes = [{"name": "partition", "column_names": [None], "unique": False}]
+        normalized_idx = BigQueryEngineSpec.normalize_indexes(indexes)
+        self.assertEqual(normalized_idx, [])
+
+        indexes = [{"name": "partition", "column_names": ["dttm"], "unique": False}]

Review comment:
       we could add another small test with `indexes = [{"name": "partition", "column_names": ["dttm", None], "unique": False}]`




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org