You are viewing a plain text version of this content. The canonical link for it is here.

Posted to notifications@superset.apache.org by "eschutho (via GitHub)" <gi...@apache.org> on 2026/10/02 16:08:03 UTC

[PR] fix(models): silence pandas silent-downcasting FutureWarning in normalize_df [superset]

eschutho opened a new pull request, #44897:
URL: https://github.com/apache/superset/pull/44897

## Decisions made that were not in the instructions

None.

## Summary

Production logs emit this on every query that reaches `ExploreMixin.normalize_df` with an object column:

```
helpers.py: FutureWarning: Downcasting behavior in `replace` is deprecated and will be removed in a future version. ... df.replace([np.inf, -np.inf], np.nan, inplace=True)
```

`normalize_df` replaced ±inf with NaN via `df.replace(..., inplace=True)`, which triggers pandas' deprecated silent downcasting on object-dtype columns. This replaces it with `df.mask(df.isin([np.inf, -np.inf])).infer_objects()`, which emits no warning and does not touch any global pandas option.

`normalize_df` is the only emitter (the only ±inf replace in `superset/`), is defined only on `ExploreMixin` with no overrides, and both of its callers use the returned frame.

No behavior change: the resulting dtypes and values are identical to the previous code (object columns of floats/ints still become float64/int64; strings stay object). The one nuance is that the input frame is no longer mutated in place; both callers already use the return value.

## Test plan

- Added `test_normalize_df_replaces_infinity_without_downcasting_warning` (fails on base with the `FutureWarning` promoted to an error, passes with the fix).
- `pytest tests/unit_tests/models/helpers_test.py` — all `normalize_df` tests pass. `test_build_like_predicate_is_case_insensitive_and_escaped` fails identically on the base commit (pre-existing, unrelated).
- `ruff check` / `ruff format --check` clean.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org

Re: [PR] fix(models): silence pandas silent-downcasting FutureWarning in normalize_df [superset]

Posted by "eschutho (via GitHub)" <gi...@apache.org>.

eschutho commented on PR #44897:
URL: https://github.com/apache/superset/pull/44897#issuecomment-5956367466

   @rebenitez1802 could you take a look at this one? It continues the pandas FutureWarning cleanup you reviewed in #40936, #42004 and #42272. Small change (`normalize_df` + one unit test). Shortcut: sc-124089.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org

Re: [PR] fix(models): silence pandas silent-downcasting FutureWarning in normalize_df [superset]

Posted by "eschutho (via GitHub)" <gi...@apache.org>.

eschutho merged PR #44897:
URL: https://github.com/apache/superset/pull/44897


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org

Re: [PR] fix(models): silence pandas silent-downcasting FutureWarning in normalize_df [superset]

Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.

bito-code-review[bot] commented on PR #44897:
URL: https://github.com/apache/superset/pull/44897#issuecomment-5956335454

   AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org

Re: [PR] fix(models): silence pandas silent-downcasting FutureWarning in normalize_df [superset]

Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.

bito-code-review[bot] commented on code in PR #44897:
URL: https://github.com/apache/superset/pull/44897#discussion_r4167653271


##########
superset/models/helpers.py:
##########
@@ -2635,7 +2635,9 @@ def normalize_df(
         if getattr(self, "enforce_numerical_metrics", True):
             dataframe_utils.df_metrics_to_num(df, query_object)
 
-        df.replace([np.inf, -np.inf], np.nan, inplace=True)
+        # ``mask`` + ``infer_objects`` rather than ``replace``, which emits a

Review Comment:
   <div>
   
   
   <div id="suggestion">
   <div id="issue"><b>Silent dtype change on object columns</b></div>
   <div id="fix">
   
   This changes object-column dtypes even when no ±inf is present: verified by execution, `mask().infer_objects()` converts object-float columns to float64 and object-string columns to `str` (pandas 3), while the old `replace` left dtypes untouched. Downstream `exec_post_processing` and chart serialization now see different dtypes. Prefer `df.where(~df.isin([np.inf, -np.inf]))`, which masks without re-typing.
   </div>
   
   
   </div>
   
   
   
   
   <small><i>Code Review Run #1f63e3</i></small>
   </div>
   
   ---
   Should Bito avoid suggestions like this for future reviews? (<a href=https://alpha.bito.ai/home/ai-agents/review-rules>Manage Rules</a>)
   - [ ] Yes, avoid them



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org

Re: [PR] fix(models): silence pandas silent-downcasting FutureWarning in normalize_df [superset]

Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.

codecov[bot] commented on PR #44897:
URL: https://github.com/apache/superset/pull/44897#issuecomment-5956684237

   ## [Codecov](https://app.codecov.io/gh/apache/superset/pull/44897?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :white_check_mark: All modified and coverable lines are covered by tests.
   :white_check_mark: Project coverage is 65.96%. Comparing base ([`0fdfd66`](https://app.codecov.io/gh/apache/superset/commit/0fdfd6660e0ac33280f6b075d8fd35ea735821b4?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`07747d3`](https://app.codecov.io/gh/apache/superset/commit/07747d39e081dbd3f1786c5a64b30d71169f5ed2?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   :warning: Report is 1 commits behind head on master.
   > :exclamation:  There is a different number of reports uploaded between BASE (0fdfd66) and HEAD (07747d3). Click for more details.
   > 
   > <details><summary>HEAD has 4 uploads less than BASE</summary>
   >
   >| Flag | BASE (0fdfd66) | HEAD (07747d3) |
   >|------|------|------|
   >|python|7|3|
   ></details>
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@             Coverage Diff             @@
   ##           master   #44897       +/-   ##
   ===========================================
   - Coverage   82.09%   65.96%   -16.13%     
   ===========================================
     Files        2995     2995               
     Lines      184988   184988               
     Branches    42814    42814               
   ===========================================
   - Hits       151872   122036    -29836     
   - Misses      30357    60270    +29913     
   + Partials     2759     2682       -77     
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/superset/pull/44897/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [hive](https://app.codecov.io/gh/apache/superset/pull/44897/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `36.30% <0.00%> (ø)` | |
   | [mysql](https://app.codecov.io/gh/apache/superset/pull/44897/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [postgres](https://app.codecov.io/gh/apache/superset/pull/44897/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [presto](https://app.codecov.io/gh/apache/superset/pull/44897/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.18% <100.00%> (ø)` | |
   | [python](https://app.codecov.io/gh/apache/superset/pull/44897/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `55.20% <100.00%> (-30.93%)` | :arrow_down: |
   | [sqlite](https://app.codecov.io/gh/apache/superset/pull/44897/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `55.07% <100.00%> (ø)` | |
   | [unit](https://app.codecov.io/gh/apache/superset/pull/44897/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   </details>
   
   [:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/superset/pull/44897?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).   
   :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   - :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org