[GitHub] [pinot] KKcorps commented on a diff in pull request #9320: Allow ingestion of errored records with incorrect datatype

[GitBox <gi...@apache.org>]

KKcorps commented on code in PR #9320:
URL: https://github.com/apache/pinot/pull/9320#discussion_r960890767


##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/recordtransformer/DataTypeTransformer.java:
##########
@@ -95,7 +103,12 @@ public GenericRow transform(GenericRow record) {
 
         record.putValue(column, value);
       } catch (Exception e) {
-        throw new RuntimeException("Caught exception while transforming data type for column: " + column, e);
+        if (!_useDefaultValueOnError) {
+          throw new RuntimeException("Caught exception while transforming data type for column: " + column, e);
+        } else {
+          LOGGER.debug("Caught exception while transforming data type for column: {}", column, e);
+          record.putValue(column, null);

Review Comment:
   Yes. Since `NullValueTransformer` takes care of this, I don't want to duplicate that logic and introduce the risk of it diverging from the current logic in future patches.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org