[GitHub] [incubator-pinot] siddharthteotia commented on a change in pull request #4745: [SegmentGeneratorConfig Cleanup] Replace checkTimeColumnValidityDuringGeneration with skipTimeValueCheck

[GitBox <gi...@apache.org>]

siddharthteotia commented on a change in pull request #4745: [SegmentGeneratorConfig Cleanup] Replace checkTimeColumnValidityDuringGeneration with skipTimeValueCheck
URL: https://github.com/apache/incubator-pinot/pull/4745#discussion_r339655556
 
 

 ##########
 File path: pinot-core/src/main/java/org/apache/pinot/core/indexsegment/generator/SegmentGeneratorConfig.java
 ##########
 @@ -112,7 +112,7 @@
   private String _simpleDateFormat = null;
   // Use on-heap or off-heap memory to generate index (currently only affect inverted index and star-tree v2)
   private boolean _onHeap = false;
-  private boolean _checkTimeColumnValidityDuringGeneration = true;
+  private boolean _skipTimeValueCheck = false;
 
 Review comment:
   If the default is false, how will we catch invalid time values during segment generation?  We will have to push this to users (outside Pinot code) who are using SegmentGeneratorConfig. Earlier, the default was true but the check was disabled for realtime and some unit tests where fixing the invalid time values required rewriting a set of Avro files that were used by tests. Still in some other tests, the check was enabled because those tests were doing data generation programmatically (and so easier to control generation of valid time values) as opposed to Avro test files.
   
   I feel the default should be true.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org