You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "fuyun2024 (via GitHub)" <gi...@apache.org> on 2023/08/13 07:36:17 UTC
[GitHub] [incubator-paimon] fuyun2024 opened a new pull request, #1806: [flink] Support dateFormat compute functions
fuyun2024 opened a new pull request, #1806:
URL: https://github.com/apache/incubator-paimon/pull/1806
Support dateFormat compute functions
### Purpose
<!-- Linking this pull request to the issue -->
Linked issue: close #1805
<!-- What is the purpose of the change -->
### Tests
<!-- List UT and IT cases to verify this change -->
### API and Format
<!-- Does this change affect API or storage format -->
### Documentation
<!-- Does this change introduce a new feature -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-paimon] JingsongLi commented on a diff in pull request #1806: [flink] Support dateFormat compute functions
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on code in PR #1806:
URL: https://github.com/apache/incubator-paimon/pull/1806#discussion_r1292918181
##########
paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/action/cdc/Expression.java:
##########
@@ -239,6 +251,36 @@ public String eval(String input) {
}
}
+ /** date format from a time input. */
+ final class DateFormat implements Expression {
+
+ private static final long serialVersionUID = 1L;
+
+ private final String fieldReference;
+ private final String pattern;
+
+ private DateFormat(String fieldReference, String pattern) {
+ this.fieldReference = fieldReference;
+ this.pattern = pattern;
+ }
+
+ @Override
+ public String fieldReference() {
+ return fieldReference;
+ }
+
+ @Override
+ public DataType outputType() {
+ return DataTypes.STRING();
+ }
+
+ @Override
+ public String eval(String input) {
+ LocalDateTime localDateTime = DateTimeUtils.toLocalDateTime(input, 0);
+ return localDateTime.format(DateTimeFormatter.ofPattern(pattern));
Review Comment:
We should reuse pattern, `DateTimeFormatter.ofPattern(pattern)` could be a class field.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-paimon] fuyun2024 commented on a diff in pull request #1806: [flink] Support dateFormat compute functions
Posted by "fuyun2024 (via GitHub)" <gi...@apache.org>.
fuyun2024 commented on code in PR #1806:
URL: https://github.com/apache/incubator-paimon/pull/1806#discussion_r1293093981
##########
paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/action/cdc/Expression.java:
##########
@@ -239,6 +251,36 @@ public String eval(String input) {
}
}
+ /** date format from a time input. */
+ final class DateFormat implements Expression {
+
+ private static final long serialVersionUID = 1L;
+
+ private final String fieldReference;
+ private final DateTimeFormatter formatter;
Review Comment:
Thank you for your suggestion, i resubmitted the code .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-paimon] JingsongLi merged pull request #1806: [flink] Support dateFormat compute functions
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi merged PR #1806:
URL: https://github.com/apache/incubator-paimon/pull/1806
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-paimon] fuyun2024 commented on a diff in pull request #1806: [flink] Support dateFormat compute functions
Posted by "fuyun2024 (via GitHub)" <gi...@apache.org>.
fuyun2024 commented on code in PR #1806:
URL: https://github.com/apache/incubator-paimon/pull/1806#discussion_r1293007296
##########
paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/action/cdc/Expression.java:
##########
@@ -37,7 +38,7 @@
public interface Expression extends Serializable {
List<String> SUPPORTED_EXPRESSION =
- Arrays.asList("year", "month", "day", "hour", "substring", "truncate");
+ Arrays.asList("year", "month", "day", "hour", "dateFormat", "substring", "truncate");
Review Comment:
@JingsongLi I resubmitted the code ， could you help to review this pull request?
##########
paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/action/cdc/Expression.java:
##########
@@ -37,7 +38,7 @@
public interface Expression extends Serializable {
List<String> SUPPORTED_EXPRESSION =
- Arrays.asList("year", "month", "day", "hour", "substring", "truncate");
+ Arrays.asList("year", "month", "day", "hour", "dateFormat", "substring", "truncate");
Review Comment:
@JingsongLi I resubmitted the code ， could you help to review this pull request?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-paimon] JingsongLi commented on a diff in pull request #1806: [flink] Support dateFormat compute functions
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on code in PR #1806:
URL: https://github.com/apache/incubator-paimon/pull/1806#discussion_r1292917784
##########
paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/action/cdc/Expression.java:
##########
@@ -37,7 +38,7 @@
public interface Expression extends Serializable {
List<String> SUPPORTED_EXPRESSION =
- Arrays.asList("year", "month", "day", "hour", "substring", "truncate");
+ Arrays.asList("year", "month", "day", "hour", "dateFormat", "substring", "truncate");
Review Comment:
maybe `date_format`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-paimon] JingsongLi commented on a diff in pull request #1806: [flink] Support dateFormat compute functions
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on code in PR #1806:
URL: https://github.com/apache/incubator-paimon/pull/1806#discussion_r1293058760
##########
paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/action/cdc/Expression.java:
##########
@@ -239,6 +251,36 @@ public String eval(String input) {
}
}
+ /** date format from a time input. */
+ final class DateFormat implements Expression {
+
+ private static final long serialVersionUID = 1L;
+
+ private final String fieldReference;
+ private final DateTimeFormatter formatter;
Review Comment:
Is this Serializable? Maybe you just need a lazy object here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org