You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2021/03/24 22:31:06 UTC
[GitHub] [incubator-pinot] liuchang0520 opened a new issue #6715: Add a flag in FieldSpec to indicate whether the _defaultNullValue is user-defined
liuchang0520 opened a new issue #6715:
URL: https://github.com/apache/incubator-pinot/issues/6715
Currently in [FieldSpec](https://github.com/apache/incubator-pinot/blob/master/pinot-spi/src/main/java/org/apache/pinot/spi/data/FieldSpec.java), [_defaultNullValue](https://github.com/apache/incubator-pinot/blob/master/pinot-spi/src/main/java/org/apache/pinot/spi/data/FieldSpec.java#L69) is set in this way:
(1) If this field has a user-defined default value, set _defaultNullValue to the user-defined one.
(2) If not, set _defaultNullValue based on fieldType and dataType.
The detail is in [getDefaultNullValue](https://github.com/apache/incubator-pinot/blob/master/pinot-spi/src/main/java/org/apache/pinot/spi/data/FieldSpec.java#L189).
As a result, we cannot distinguish whether the default value is set from user or not. Adding a flag(e.g. isUserDefinedDefaultValue) can resolve this issue.
[_stringDefaultNullValue](https://github.com/apache/incubator-pinot/blob/master/pinot-spi/src/main/java/org/apache/pinot/spi/data/FieldSpec.java#L182) is set based on the user-defined default value, but it is a transient field.
This change is required for integration [Null Value Support
](https://docs.pinot.apache.org/developers/advanced/null-value-support) with our internal Pinot streaming message decoder. Previously we set the null field to be default value. After the change, we want to use this flag when the null value appears in one field:
(1) If it has a user-defined default value, set the result as the user-defined one.
(2) If not, set the result as null.
CC @chenboat and @Jackie-Jiang as they are aware of this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] kishoreg commented on issue #6715: Add a flag in FieldSpec to indicate whether the _defaultNullValue is user-defined
Posted by GitBox <gi...@apache.org>.
kishoreg commented on issue #6715:
URL: https://github.com/apache/incubator-pinot/issues/6715#issuecomment-806228669
> As a result, we cannot distinguish whether the default value is set from user or not. Adding a flag(e.g. isUserDefinedDefaultValue) can resolve this issue.
We can know this by looking at the schema right? why do we need another flag for this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] Jackie-Jiang commented on issue #6715: Add a flag in FieldSpec to indicate whether the _defaultNullValue is user-defined
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on issue #6715:
URL: https://github.com/apache/incubator-pinot/issues/6715#issuecomment-806231068
You don't need another flag in the `FieldSpec`, instead you can add a method in `FieldSpec` which is json-ignored and return whether the `_stringDefaultNullValue` is `null` (whether the default is user-defined).
I don't quite follow why you need to know whether the default is user-defined though. You can always leave null field as is and Pinot will fill it with default (either user-defined or not)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] liuchang0520 commented on issue #6715: Add a flag in FieldSpec to indicate whether the _defaultNullValue is user-defined
Posted by GitBox <gi...@apache.org>.
liuchang0520 commented on issue #6715:
URL: https://github.com/apache/incubator-pinot/issues/6715#issuecomment-806235336
@kishoreg @Jackie-Jiang OK make sense. I missed the step where Pinot fills in the default value after getting a null field from decoder.
This change is not needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] liuchang0520 closed issue #6715: Add a flag in FieldSpec to indicate whether the _defaultNullValue is user-defined
Posted by GitBox <gi...@apache.org>.
liuchang0520 closed issue #6715:
URL: https://github.com/apache/incubator-pinot/issues/6715
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org