[GitHub] sumedhsakdeo opened a new pull request #5632: Fetch a batch of rows from bigquery

[GitBox <gi...@apache.org>]

sumedhsakdeo opened a new pull request #5632: Fetch a batch of rows from bigquery
URL: https://github.com/apache/incubator-superset/pull/5632
 
 
   While running superset with Google BigQuery as the database I found that the queries are very slow. To fetch 1000 rows, it was taking approx. ~2 minutes. On further investigation I found that, they way our cursor is configured, it makes an REST API call for every row fetched, instead of one API call to fetch a batch of rows. pybigquery handles this batch fetch configuration, however, it does not work with cursor from the raw_connection used in superset.  
   
   After my change, the query to fetch 1000 rows from BigQuery takes ~1.5 seconds. Which is on par with the query runtime, when run from BigQuery query editor.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org