You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2021/10/20 14:46:05 UTC
[GitHub] [pinot] etadelta222 opened a new issue #7606: 'null' value support udf
etadelta222 opened a new issue #7606:
URL: https://github.com/apache/pinot/issues/7606
I have a date of birth (dob) column in my source postgres table. I'm creating a realtime table in Pinot and using datatype:timestamp in my schema but this ends up defaulting the 'null' dob values to '1970-01-01' which is incorrect. So instead I'm using datatype:string and using the transformConfigs in table config to convert it.
Below is what I've tried out.
Error on the 'null' values when trying to use the 'todatetime' function.
`,
"ingestionConfig": {
"transformConfigs": [
{
"columnName": "dob",
"transformFunction": "toDateTime(date_of_birth,'yyyy-MM-dd')"
}
]
},`
Throws this error:
`java.lang.IllegalStateException: Caught exception while invoking method: public static java.lang.String org.apache.pinot.common.function.scalar.DateTimeFunctions.toDateTime(long,java.lang.String) with arguments: [null, yyyy-MM-dd]`
Attempted using Groovy but it does not work with Pinot builtin functions.
` "ingestionConfig": {
"transformConfigs": [
{
"columnName": "dob",
"transformFunction": "Groovy({date_of_birth != 'null' ? toDateTime(date_of_birth,'yyyy-MM-dd'): '1900-01-01'},date_of_birth)"
}
]
},`
Give this error:
`groovy.lang.MissingMethodException: No signature of method: Script1.toDateTime() is applicable for argument types: (java.lang.Long, java.lang.String) values: [304992000000, yyyy-MM-dd]`
Request is to add null support udf.
Pinot version: 0.8.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on issue #7606: 'null' value support udf
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on issue #7606:
URL: https://github.com/apache/pinot/issues/7606#issuecomment-948108293
One workaround would be using `TIMESTAMP` and put default value as `1900-01-01 00:00:00`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] etadelta222 commented on issue #7606: 'null' value support udf
Posted by GitBox <gi...@apache.org>.
etadelta222 commented on issue #7606:
URL: https://github.com/apache/pinot/issues/7606#issuecomment-949044533
Thanks Jackie, I'll give this a shot. I was able to use the following as a workaround:
`"ingestionConfig": {
"transformConfigs": [
{
"columnName": "dob",
"transformFunction": "Groovy({date_of_birth != 'null' ? new Date(date_of_birth).format('yyyy-MM-dd'):'1900-01-01'},date_of_birth)"
}
]
},`
I'd prefer not to create a new column just to show a formatted date.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] etadelta222 commented on issue #7606: 'null' value support udf
Posted by GitBox <gi...@apache.org>.
etadelta222 commented on issue #7606:
URL: https://github.com/apache/pinot/issues/7606#issuecomment-969265326
Adding the default value worked @Jackie-Jiang!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on issue #7606: 'null' value support udf
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on issue #7606:
URL: https://github.com/apache/pinot/issues/7606#issuecomment-969270029
@etadelta222 Thanks for confirming it! Shall we close the issue if the problem is solved?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org