[GitHub] [superset] ktmud commented on a change in pull request #16450: fix(query): handle Infinity in query results

[GitBox <gi...@apache.org>]

ktmud commented on a change in pull request #16450:
URL: https://github.com/apache/superset/pull/16450#discussion_r696047946



##########
File path: superset/common/query_context.py
##########
@@ -235,8 +239,6 @@ def normalize_df(self, df: pd.DataFrame, query_object: QueryObject) -> pd.DataFr
         if self.enforce_numerical_metrics:
             self.df_metrics_to_num(df, query_object)
 
-        df.replace([np.inf, -np.inf], np.nan, inplace=True)

Review comment:
       Infs will be forced into `null`s by JSON encoder anyway.

##########
File path: superset/common/query_context.py
##########
@@ -276,11 +278,19 @@ def get_query_result(self, query_object: QueryObject) -> QueryResult:
     @staticmethod
     def df_metrics_to_num(df: pd.DataFrame, query_object: QueryObject) -> None:
         """Converting metrics to numeric when pandas.read_sql cannot"""
+        metric_names: List[str] = query_object.metric_names
         for col, dtype in df.dtypes.items():
-            if dtype.type == np.object_ and col in query_object.metric_names:
+            if dtype.type == np.object_ and col in metric_names:
                 # soft-convert a metric column to numeric
                 # will stay as strings if conversion fails
-                df[col] = df[col].infer_objects()
+                df[col] = pd.to_numeric(
+                    df[col]
+                    .replace(INFINITY_LITERALS, np.inf)
+                    .replace(NEGATIVE_INFINITY_LITERALS, -np.inf),

Review comment:
       Some DBAPI (e.g. Druid) returns `Infinity` as quoted strings `"Infinity"`, we must manually replace them into Numpy infs.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org