You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2020/05/22 13:11:22 UTC
[GitHub] [incubator-hudi] nsivabalan commented on a change in pull request #1648: [HUDI-916]: added support for multiple input formats in TimestampBasedKeyGenerator
nsivabalan commented on a change in pull request #1648:
URL: https://github.com/apache/incubator-hudi/pull/1648#discussion_r429230492
##########
File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/keygen/TimestampBasedKeyGenerator.java
##########
@@ -53,44 +56,74 @@
private final TimestampType timestampType;
- private SimpleDateFormat inputDateFormat;
-
private final String outputDateFormat;
+ private DateTimeFormatter inputFormatter;
+ private final String configInputDateFormatList;
+ private final String configInputDateFormatDelimiter;
// TimeZone detailed settings reference
// https://docs.oracle.com/javase/8/docs/api/java/util/TimeZone.html
- private final TimeZone timeZone;
+ private final DateTimeZone inputDateTimeZone;
+ private final DateTimeZone outputDateTimeZone;
/**
* Supported configs.
*/
static class Config {
// One value from TimestampType above
- private static final String TIMESTAMP_TYPE_FIELD_PROP = "hoodie.deltastreamer.keygen.timebased.timestamp.type";
- private static final String INPUT_TIME_UNIT =
+ public static final String TIMESTAMP_TYPE_FIELD_PROP = "hoodie.deltastreamer.keygen.timebased.timestamp.type";
+ public static final String INPUT_TIME_UNIT =
"hoodie.deltastreamer.keygen.timebased.timestamp.scalar.time.unit";
- private static final String TIMESTAMP_INPUT_DATE_FORMAT_PROP =
+ //This prop can now accept list of input date formats.
+ public static final String TIMESTAMP_INPUT_DATE_FORMAT_PROP =
"hoodie.deltastreamer.keygen.timebased.input.dateformat";
- private static final String TIMESTAMP_OUTPUT_DATE_FORMAT_PROP =
+ public static final String TIMESTAMP_INPUT_DATE_FORMAT_LIST_DELIMITER_REGEX_PROP = "hoodie.deltastreamer.keygen.timebased.input.dateformatlistdelimiterregex";
Review comment:
can we add dots (".") between words. hoodie.deltastreamer.keygen.timebased.input.dateformat.list.delimiter.regex
##########
File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/keygen/TimestampBasedKeyGenerator.java
##########
@@ -53,44 +56,74 @@
private final TimestampType timestampType;
- private SimpleDateFormat inputDateFormat;
-
private final String outputDateFormat;
+ private DateTimeFormatter inputFormatter;
+ private final String configInputDateFormatList;
+ private final String configInputDateFormatDelimiter;
// TimeZone detailed settings reference
// https://docs.oracle.com/javase/8/docs/api/java/util/TimeZone.html
- private final TimeZone timeZone;
+ private final DateTimeZone inputDateTimeZone;
+ private final DateTimeZone outputDateTimeZone;
/**
* Supported configs.
*/
static class Config {
// One value from TimestampType above
- private static final String TIMESTAMP_TYPE_FIELD_PROP = "hoodie.deltastreamer.keygen.timebased.timestamp.type";
- private static final String INPUT_TIME_UNIT =
+ public static final String TIMESTAMP_TYPE_FIELD_PROP = "hoodie.deltastreamer.keygen.timebased.timestamp.type";
Review comment:
why public? package private isn't sufficient?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org