[GitHub] agrawaldevesh opened a new pull request #6719: Adding support for Pinot

[GitBox <gi...@apache.org>]

agrawaldevesh opened a new pull request #6719: Adding support for Pinot
URL: https://github.com/apache/incubator-superset/pull/6719
 
 
   Summary: Added limited support for visualizations with Pinot via
   Sqlalchemy.
   
   Pinot QL (PQL) is a bit weird and limited, and this patch hacks superset to
   deal with that weirdness:
   
   1. PQL does not support two constant comparisons. For example it won't
   support X >= 12 and X <= 15. One must use BETWEEN instead.
   
   2. Pinot's grouping by time is best done as a long epoch. Grouping by a
   time string is really slow and times out.
   
   3. Pinot's response does not respect column aliases. So columns are not
   named what they are expected to. So we remember the given column aliases
   and then stamp them back onto the dataframe
   
   4. Pinot's Json rest call does not return the output types. Instead
   everything is cast to string. So when grouping by time, the group key
   is integral and has to be treated specially when casting back to the
   dataframe __timestamp column.
   
   5. Finally, pinot does support grouping by on expressions. But those
   expressions cannot then appear on the select clause. They are returned
   regardless in the response. ie, 'select foo, count(*) from bar group by
   foo' is okay, but 'select expr(foo), count(*) from bar group by
   expr(foo)' ain't. One must use 'select count(*) from bar group by
   expr(foo)'.
   
   I also fixed a couple of things that looked like bugs to me: for
   example, the row-ordering-limit should come at the end always.
   
   Test Plan: Tested with the modified pinotdb sqlalchemy driver and an
   internal pinot cluster. The pinotdb driver changes are in
   https://github.com/agrawaldevesh/pinot-dbapi.
   
   Pinot does not support orderby-limit for aggregated queries. To annotate
   a query as an aggregate query, this patch adds a hint to the prepared
   select statement that the pinotdb sqlalchemy driver then heeds.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org