You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2020/05/18 20:19:06 UTC
[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #5399: DATE_TIME should work as the primary time column for Pinot tables
Jackie-Jiang commented on a change in pull request #5399:
URL: https://github.com/apache/incubator-pinot/pull/5399#discussion_r426869167
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/segment/name/NormalizedDateSegmentNameGenerator.java
##########
@@ -60,16 +61,16 @@ public NormalizedDateSegmentNameGenerator(String tableName, @Nullable String seg
}
_outputSDF.setTimeZone(TimeZone.getTimeZone("UTC"));
- // Parse input time format: 'EPOCH' or 'SIMPLE_DATE_FORMAT:<pattern>'
- if (Preconditions.checkNotNull(timeFormat).equals(TimeFormat.EPOCH.toString())) {
- _inputTimeUnit = timeType;
+ // Parse input time format: 'EPOCH' or 'SIMPLE_DATE_FORMAT' using pattern
+ Preconditions.checkNotNull(dateTimeFormatSpec);
+ TimeFormat timeFormat = dateTimeFormatSpec.getTimeFormat();
+ if (timeFormat.equals(TimeFormat.EPOCH)) {
Review comment:
```suggestion
if (timeFormat == TimeFormat.EPOCH) {
```
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/SegmentColumnarIndexCreator.java
##########
@@ -362,12 +362,14 @@ private void writeMetadata()
properties.setProperty(DIMENSIONS, config.getDimensions());
properties.setProperty(METRICS, config.getMetrics());
properties.setProperty(DATETIME_COLUMNS, config.getDateTimeColumnNames());
- properties.setProperty(TIME_COLUMN_NAME, config.getTimeColumnName());
+ String timeColumnName = config.getTimeColumnName();
+ if (timeColumnName != null && !config.getSchema().getDateTimeNames().contains(timeColumnName)) {
Review comment:
Why do we need the second part of the check? What if the schema has no DateTimeField but only TimeField?
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/SegmentColumnarIndexCreator.java
##########
@@ -362,12 +362,14 @@ private void writeMetadata()
properties.setProperty(DIMENSIONS, config.getDimensions());
properties.setProperty(METRICS, config.getMetrics());
properties.setProperty(DATETIME_COLUMNS, config.getDateTimeColumnNames());
- properties.setProperty(TIME_COLUMN_NAME, config.getTimeColumnName());
+ String timeColumnName = config.getTimeColumnName();
+ if (timeColumnName != null && !config.getSchema().getDateTimeNames().contains(timeColumnName)) {
+ properties.setProperty(TIME_COLUMN_NAME, timeColumnName);
+ }
properties.setProperty(SEGMENT_TOTAL_DOCS, String.valueOf(totalDocs));
// Write time related metadata (start time, end time, time unit)
- String timeColumn = config.getTimeColumnName();
- ColumnIndexCreationInfo timeColumnIndexCreationInfo = indexCreationInfoMap.get(timeColumn);
+ ColumnIndexCreationInfo timeColumnIndexCreationInfo = indexCreationInfoMap.get(timeColumnName);
Review comment:
Check whether `timeColumnName` is `null` first
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org