[GitHub] [superset] TColl commented on pull request #12806: refactor: vectorise big integer to string type conversion

[GitBox <gi...@apache.org>]

TColl commented on pull request #12806:
URL: https://github.com/apache/superset/pull/12806#issuecomment-773874994


   hi @ktmud, thanks for picking this up!
   
   I think I missed something in my original profiling; I've just checked again on my side and I'm seeing the same results you are, across small and large dataframes and varying fractions of large integers. 
   
   I'll admit I'm used to the 'optimisation' I proposed generally being orders of magnitude faster than manual iteration over each value within a dataframe so I was guilty of confirmation bias when I reviewed my original, mistaken benchmarking, but the context of converting the df to a list of dicts and keeping it in that format is the dominant factor here and as such I'm happy to close this out at this stage (unless anyone can think of a better way to handle this - maybe a more performant alternative to pandas `DataFrame.to_dict(orient="records")`?)


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org