You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by GitBox <gi...@apache.org> on 2022/08/15 16:46:16 UTC
[GitHub] [superset] whynotbrooklyn opened a new issue, #21093: Cumsum Creating Discontinuous Lines
whynotbrooklyn opened a new issue, #21093:
URL: https://github.com/apache/superset/issues/21093
A clear and concise description of what the bug is.
#### How to reproduce the bug
1. Create a cumsum with multiple series where some series do not have all of the dates
2. Plot
### Expected results
I would expect that when there are gaps in the date data, that the superset cumsum function creates a continuous line filling with the last value
### Actual results
Lines break when one series has a data point and the others don't.
#### Screenshots
### Environment
- Chrome latest version
- superset version: 1.5.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] sfirke commented on issue #21093: Cumsum Creating Discontinuous Lines
Posted by "sfirke (via GitHub)" <gi...@apache.org>.
sfirke commented on issue #21093:
URL: https://github.com/apache/superset/issues/21093#issuecomment-1410725166
Hi there - it's not a function of Superset I'd ever used before nor do I know its codebase. So I can't speak to this, sorry, But it seems like a well-done bug report.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] whynotbrooklyn commented on issue #21093: Cumsum Creating Discontinuous Lines
Posted by GitBox <gi...@apache.org>.
whynotbrooklyn commented on issue #21093:
URL: https://github.com/apache/superset/issues/21093#issuecomment-1215559338
I think it's related to this bug that was fixed for single series charts: https://github.com/apache/superset/issues/1661
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] whynotbrooklyn commented on issue #21093: Cumsum Creating Discontinuous Lines
Posted by "whynotbrooklyn (via GitHub)" <gi...@apache.org>.
whynotbrooklyn commented on issue #21093:
URL: https://github.com/apache/superset/issues/21093#issuecomment-1410779724
> Hi there - it's not a function of Superset I'd ever used before nor do I know its codebase. So I can't speak to this, sorry, But it seems like a well-done bug report.
Well thanks for taking a look at the ticket anyway! Sorry to randomly tag; we're getting pretty desperate to get this fixed. Our client just asked for the cumulative charts in weekly rather than monthly increments - which only makes this bug look worse.
Given that this has already been fixed for single series charts, I'm hoping it won't be an intensive to apply the same fix for the multi-series version. I'd be happy to even take a stab at it myself if anyone could point me to the right file location.
Adding in @lilykuang @hughhhh and @michael-s-molina for expanded visibility.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] ryandawsonuk commented on issue #21093: Cumsum Creating Discontinuous Lines
Posted by GitBox <gi...@apache.org>.
ryandawsonuk commented on issue #21093:
URL: https://github.com/apache/superset/issues/21093#issuecomment-1357365555
I'm also seeing the same problem with different styles of presenting the chart - seeing with both line and bar
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] daro-123 commented on issue #21093: Cumsum Creating Discontinuous Lines
Posted by "daro-123 (via GitHub)" <gi...@apache.org>.
daro-123 commented on issue #21093:
URL: https://github.com/apache/superset/issues/21093#issuecomment-1735138964
I have this issue as well where there is no continuous line on some dates when using cumsum but resample does the trick and help to fill in the gap.
**Before**
**After**
My chart time grain is week; I'm on 2.1.1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Cumsum Creating Discontinuous Lines [superset]
Posted by "michael-s-molina (via GitHub)" <gi...@apache.org>.
michael-s-molina closed issue #21093: Cumsum Creating Discontinuous Lines
URL: https://github.com/apache/superset/issues/21093
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] rusackas commented on issue #21093: Cumsum Creating Discontinuous Lines
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on issue #21093:
URL: https://github.com/apache/superset/issues/21093#issuecomment-1410965442
You can *probably* resolve this (though I don't have a dataset to repro handy) by using the Zero Imputation option in the Fill Method control of the Advanced Analytics section of chart controls. This will take any null datapoints and convert them to 0 values, which I would _assume_ means you'll get a contiguous line thanks to the now-contiguous dataset.
<img width="316" alt="image" src="https://user-images.githubusercontent.com/812905/215866023-b7afbe0d-5b3c-43cc-aeba-bbb5c1e5f67c.png">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] sfirke commented on issue #21093: Cumsum Creating Discontinuous Lines
Posted by "sfirke (via GitHub)" <gi...@apache.org>.
sfirke commented on issue #21093:
URL: https://github.com/apache/superset/issues/21093#issuecomment-1426156892
A week later and now I'm facing this bug too. If this is the same issue it's not a matter of cumsum specifically. In my case I'm seeing zeroes correctly imputed for intervals when no dimension group has a record, but if dimension group _does_ have a record, the others don't get the zeroes. Look at Feb 7th in the table and the chart of this screenshot:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] whynotbrooklyn commented on issue #21093: Cumsum Creating Discontinuous Lines
Posted by "whynotbrooklyn (via GitHub)" <gi...@apache.org>.
whynotbrooklyn commented on issue #21093:
URL: https://github.com/apache/superset/issues/21093#issuecomment-1470547935
This one is hardly even a line graph anymore...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Cumsum Creating Discontinuous Lines [superset]
Posted by "diopol (via GitHub)" <gi...@apache.org>.
diopol commented on issue #21093:
URL: https://github.com/apache/superset/issues/21093#issuecomment-1751713150
> The fix is probably very simple. In superset\viz.py in this line
>
> https://github.com/apache/superset/blob/46a0a6e08af1b66bb617d5ebe1324b77251aee15/superset/viz.py#L232
>
>
> the nan values should be replaced with 0.
> `df = df.fillna(0).cumsum()`
> I can't easily test it, but the behavior is well documented: https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.cumsum.html
this fix works
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] lchavas commented on issue #21093: Cumsum Creating Discontinuous Lines
Posted by "lchavas (via GitHub)" <gi...@apache.org>.
lchavas commented on issue #21093:
URL: https://github.com/apache/superset/issues/21093#issuecomment-1725237575
@WulfH thank you, my cumsum dimensions (sales subdivided into stacked bars per family) were always truncated after the last value in the DataFrame, which resulted in stacked subdivisions not matching the total.
fillna(0) fixed my sums and the subtotals now match the main line, it's a beautiful sight to behold. This should be pulled
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Cumsum Creating Discontinuous Lines [superset]
Posted by "JohnDietrich-Pepper (via GitHub)" <gi...@apache.org>.
JohnDietrich-Pepper commented on issue #21093:
URL: https://github.com/apache/superset/issues/21093#issuecomment-1963054774
Ah this fix really broke our charting. We wanted to show discontinuous lines when the data is null because the data is null and NOT zero particularly when the time period hasn't occurred yet. Now Superset is assuming all null data is zero which is not the case.
@rusackas @villebro
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] whynotbrooklyn commented on issue #21093: Cumsum Creating Discontinuous Lines
Posted by "whynotbrooklyn (via GitHub)" <gi...@apache.org>.
whynotbrooklyn commented on issue #21093:
URL: https://github.com/apache/superset/issues/21093#issuecomment-1409158621
Hi [rusackas](https://github.com/rusackas) is there any way to get some traction on this? My company is trying to utilize Superset for a commercial offering and this is a pretty glaring issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] WulfH commented on issue #21093: Cumsum Creating Discontinuous Lines
Posted by "WulfH (via GitHub)" <gi...@apache.org>.
WulfH commented on issue #21093:
URL: https://github.com/apache/superset/issues/21093#issuecomment-1699328601
The fix is probably very simple. In superset\viz.py in this line https://github.com/apache/superset/blob/46a0a6e08af1b66bb617d5ebe1324b77251aee15/superset/viz.py#L232
the nan values should be replaced with 0.
`df = df.fillna(0).cumsum()`
I can't easily test it, but the behavior is well documented: https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.cumsum.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] HeathLee commented on issue #21093: Cumsum Creating Discontinuous Lines
Posted by "HeathLee (via GitHub)" <gi...@apache.org>.
HeathLee commented on issue #21093:
URL: https://github.com/apache/superset/issues/21093#issuecomment-1568602316
getting same issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] whynotbrooklyn commented on issue #21093: Cumsum Creating Discontinuous Lines
Posted by "whynotbrooklyn (via GitHub)" <gi...@apache.org>.
whynotbrooklyn commented on issue #21093:
URL: https://github.com/apache/superset/issues/21093#issuecomment-1411053189
Hi @rusackas , thanks for getting back to me!
Unfortunately, I have tried every one of the "FILL METHOD" options and none of them resolve the NAs in the subsequent output. (You can see the NAs in the table below the chart in my example pic.) It makes sense that this wouldn't have much effect given that there is no resampling done in my data. However, adding any resampling rules just produced errors so I'm not sure that is the answer either. How was this solved in the single series example that I referenced?
I have mocked up some demo data for you so that this can be better triaged: [Discontinuous Cumsum Demo Data](https://github.com/apache/superset/files/10550914/2023-01-31.2_41pm.csv)
The options that I have selected are simply:
X-Axis: event_date
Metric: COUNT(*)
Dimensions: grouping_variable
Rolling Function: cumsum
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] rusackas commented on issue #21093: Cumsum Creating Discontinuous Lines
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on issue #21093:
URL: https://github.com/apache/superset/issues/21093#issuecomment-1414461496
Thank you for the example dataset! Sadly, I was indeed able to validate the issue you're experiencing. I'll tag a few people who might take interest in this if/when time allows, but I'd love to see _anyone_ give this issue some help.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] sandeepson commented on issue #21093: Cumsum Creating Discontinuous Lines
Posted by "sandeepson (via GitHub)" <gi...@apache.org>.
sandeepson commented on issue #21093:
URL: https://github.com/apache/superset/issues/21093#issuecomment-1470602903
I am getting the same issue for the line chart. Following this thread for a resolution. Please assist!!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] whynotbrooklyn commented on issue #21093: Cumsum Creating Discontinuous Lines
Posted by "whynotbrooklyn (via GitHub)" <gi...@apache.org>.
whynotbrooklyn commented on issue #21093:
URL: https://github.com/apache/superset/issues/21093#issuecomment-1726016978
@rusackas is there anything you can do to get this change pulled into main?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] zhaoyongjie commented on issue #21093: Cumsum Creating Discontinuous Lines
Posted by "zhaoyongjie (via GitHub)" <gi...@apache.org>.
zhaoyongjie commented on issue #21093:
URL: https://github.com/apache/superset/issues/21093#issuecomment-1732055177
This seems to be a new feature,
It's not the same as the 'resample' operation. '[Resample](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.resample.html)' refers to filling in rows with missing time indices, whereas this feature requires modifying the original Dataframe which means replace some values with specific values, e.g. null -> 0.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] whynotbrooklyn commented on issue #21093: Cumsum Creating Discontinuous Lines
Posted by "whynotbrooklyn (via GitHub)" <gi...@apache.org>.
whynotbrooklyn commented on issue #21093:
URL: https://github.com/apache/superset/issues/21093#issuecomment-1470540858
@yousoph @zhaoyongjie @villebro Any update on when this can get fixed?
We're supposed to be going live with this soon and this graph is pretty bad... especially when you notice that without markers, you can't even tell that there's data for May in the control group...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Cumsum Creating Discontinuous Lines [superset]
Posted by "diopol (via GitHub)" <gi...@apache.org>.
diopol commented on issue #21093:
URL: https://github.com/apache/superset/issues/21093#issuecomment-1751775765
this change worked for me:
https://github.com/apache/superset/blob/46a0a6e08af1b66bb617d5ebe1324b77251aee15/superset/utils/pandas_postprocessing/cum.py#L56
changed to:
```python
df_cum = _append_columns(df, getattr(df_cum.fillna(0), operation)(), columns)
```
I'm using `3.0.0rc3`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Cumsum Creating Discontinuous Lines [superset]
Posted by "villebro (via GitHub)" <gi...@apache.org>.
villebro commented on issue #21093:
URL: https://github.com/apache/superset/issues/21093#issuecomment-1882486445
FYI here's a fix for this issue: #26429
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org