You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by GitBox <gi...@apache.org> on 2020/02/06 18:50:06 UTC
[GitHub] [incubator-superset] robdiciuccio opened a new pull request #9096:
SQL Lab: Use numpy structured arrays, fallback to JSON serialization
robdiciuccio opened a new pull request #9096: SQL Lab: Use numpy structured arrays, fallback to JSON serialization
URL: https://github.com/apache/incubator-superset/pull/9096
### CATEGORY
Choose one
- [x] Bug Fix
- [ ] Enhancement (new features, refinement)
- [ ] Refactor
- [ ] Add tests
- [ ] Build / Development Environment
- [ ] Documentation
### SUMMARY
Some SQL Lab queries were failing due to deeply nested data structures in columns. Additionally, `numpy` was sometimes getting confused when only one row was returned containing a single column of nested data. This PR 1) leverages `numpy` [structured arrays](https://docs.scipy.org/doc/numpy/user/basics.rec.html) to improve columnar data access, and 2) attempts JSON serialization of nested data structures that PyArrow can't handle natively.
### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
### TEST PLAN
Test against various database backends containing columns with nested data.
### ADDITIONAL INFORMATION
<!--- Check any relevant boxes with "x" -->
<!--- HINT: Include "Fixes #nnn" if you are fixing an existing issue -->
- [ ] Has associated issue:
- [ ] Changes UI
- [ ] Requires DB Migration.
- [ ] Confirm DB Migration upgrade and downgrade tested.
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
### REVIEWERS
@john-bodley @graceguo-supercat @michellethomas @villebro @dpgaspar @craig-rueda @mistercrunch
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [incubator-superset] serenajiang commented on issue #9096: SQL Lab:
Use numpy structured arrays, fallback to JSON serialization
Posted by GitBox <gi...@apache.org>.
serenajiang commented on issue #9096: SQL Lab: Use numpy structured arrays, fallback to JSON serialization
URL: https://github.com/apache/incubator-superset/pull/9096#issuecomment-584449351
@robdiciuccio
FYI, we're having an issue where certain nested presto structs fail to render in sqllab. The error is: `an integer is required (got type str)`
I'm having trouble figuring out the exact situation that results in this problem, but it appears to be a regression, since this was fine a week ago. The column that errors has type:
```
MAP<STRING,STRUCT<A:BIGINT,B:STRING>>
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [incubator-superset] john-bodley merged pull request #9096: SQL
Lab: Use numpy structured arrays, fallback to JSON serialization
Posted by GitBox <gi...@apache.org>.
john-bodley merged pull request #9096: SQL Lab: Use numpy structured arrays, fallback to JSON serialization
URL: https://github.com/apache/incubator-superset/pull/9096
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org