[GitHub] michellethomas opened a new issue #5532: Dashboard position_json truncates if dashboard gets too large

[GitBox <gi...@apache.org>]

michellethomas opened a new issue #5532: Dashboard position_json truncates if dashboard gets too large
URL: https://github.com/apache/incubator-superset/issues/5532
 
 
   If a v2 dashboard gets too large, the position_json data will get truncated because the text field in mysql only allows up to 65kb of data. This will cause an internal server error as the position_json is invalid, and the data truncated will likely be unrecoverable unless you make frequent backups. More metadata is being stored in position_json in v2 (nested tabs, rows, columns), causing this issue.
   
   ### Solutions
   
   Simple changes to do soon:
   - Removing unnecessary text from position_json (like DASHBOARD_* being used in type descriptions)
   - Stop pretty printing json that goes into the db
   
   Options to consider doing:
   - Changing the type for position_json so it can store more data
   - Gzip results or otherwise compressing the data
   - Prevent the user from adding to dashboards over the size limit
   
   Option 4 is an easy change that can reduce the size from 65kb to ~45kb, so we should do this soon. I think we also want to do option 2 to reduce space. We still need to look into whether option 3 is viable. 
   
   - [x] I have checked the superset logs for python stacktraces and included it here as text if any
   - [x] I have reproduced the issue with at least the latest released version of superset
   - [x] I have checked the issue tracker for the same issue and I haven't found one similar
   
   
   ### Steps to reproduce
   
   Create a dashboard where the position_json is larger than 65kb.
   Load the dashboard and see internal server error because of invalid json
   
   @graceguo-supercat @mistercrunch @john-bodley 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org