[GitHub] [incubator-pinot] reallocf edited a comment on issue #5313: Add date time transform functions

[GitBox <gi...@apache.org>]

reallocf edited a comment on issue #5313:
URL: https://github.com/apache/incubator-pinot/issues/5313#issuecomment-623023872


   Chatted with @npawar a bit over Slack. She added a number of the functions here: https://github.com/apache/incubator-pinot/pull/5324/files so I'm focusing mostly on LocalDateTime conversions here.
   
   The interfaces should be something like:
   ```
   LocalDateTime toLocalDateTime(Long millis, String pattern)
   
   Long fromLocalDateTime(LocalDateTime localDateTime)
   ```
   
   The core complexity here is in not creating a new SimpleDateFormatter for each row (as Neha already mentioned). Right now, all DateTimeFunctions are static - if we were to write similar static functions we would either need to create a new SimpleDateFormatter for each row (too slow) or create a simple cache for SimpleDateFormatters (the joys of cache invalidation...). So, the better approach is to refactor the FunctionRegistry to allow non-static functions to be processed so we can have some shared memory between rows.
   
   I'm exploring the best way to go about this now, using the usage of TransformFunctions as an example.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org