You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2022/10/09 21:14:58 UTC
[GitHub] [iceberg] rdblue opened a new pull request, #5944: API: Update expression sanitization for relative dates and times
rdblue opened a new pull request, #5944:
URL: https://github.com/apache/iceberg/pull/5944
This updates expression sanitization to produce relative date/time strings rather than generic `(date)` and `(timestamp)` literals.
For dates within the last 90 days, the sanitized values are shown in days relative to the current date: `(date-today)`, `(date-N-days-ago)`, and `(date-N-days-from-now)`. For timestamps, the sanitized values are shown relative to the current timestamp in hours or in days: `(timestamp-about-now)` within 5 minutes, `(timestamp-N-hours-ago)` within 3 days, and `(timestamp-N-days-from-now)` within 90 days.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] nastra commented on a diff in pull request #5944: API: Update expression sanitization for relative dates and times
Posted by GitBox <gi...@apache.org>.
nastra commented on code in PR #5944:
URL: https://github.com/apache/iceberg/pull/5944#discussion_r990983232
##########
api/src/main/java/org/apache/iceberg/expressions/ExpressionUtil.java:
##########
@@ -111,7 +122,15 @@ public static boolean selectsPartitions(
private static class ExpressionSanitizer
extends ExpressionVisitors.ExpressionVisitor<Expression> {
- private static final ExpressionSanitizer INSTANCE = new ExpressionSanitizer();
+ private final long now;
+ private final int today;
+
+ private ExpressionSanitizer() {
+ long nowMillis = System.currentTimeMillis();
+ OffsetDateTime nowDateTime = Instant.ofEpochMilli(nowMillis).atOffset(ZoneOffset.UTC);
+ this.now = nowMillis * 1000;
Review Comment:
I think it would be good to name this `nowMicros` here and in the other places to make it clearer that we're dealing with microseconds
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] XBaith commented on a diff in pull request #5944: API: Update expression sanitization for relative dates and times
Posted by GitBox <gi...@apache.org>.
XBaith commented on code in PR #5944:
URL: https://github.com/apache/iceberg/pull/5944#discussion_r995530184
##########
api/src/main/java/org/apache/iceberg/expressions/ExpressionUtil.java:
##########
@@ -33,11 +38,17 @@
public class ExpressionUtil {
private static final Function<Object, Integer> HASH_FUNC =
Transforms.bucket(Integer.MAX_VALUE).bind(Types.StringType.get());
+ private static final OffsetDateTime EPOCH = Instant.ofEpochSecond(0).atOffset(ZoneOffset.UTC);
+ private static final long FIVE_MINUTES_IN_MICROS = TimeUnit.MINUTES.toMicros(5);
+ private static final long THREE_DAYS_IN_HOURS = TimeUnit.DAYS.toHours(3);
+ private static final long NINETY_DAYS_IN_HOURS = TimeUnit.DAYS.toHours(90);
private static final Pattern DATE = Pattern.compile("\\d\\d\\d\\d-\\d\\d-\\d\\d");
Review Comment:
regex can be simplified to to `\\d{4}-\\d{2}-\\d{2}`, the following expression is the same
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] XBaith commented on a diff in pull request #5944: API: Update expression sanitization for relative dates and times
Posted by GitBox <gi...@apache.org>.
XBaith commented on code in PR #5944:
URL: https://github.com/apache/iceberg/pull/5944#discussion_r995530184
##########
api/src/main/java/org/apache/iceberg/expressions/ExpressionUtil.java:
##########
@@ -33,11 +38,17 @@
public class ExpressionUtil {
private static final Function<Object, Integer> HASH_FUNC =
Transforms.bucket(Integer.MAX_VALUE).bind(Types.StringType.get());
+ private static final OffsetDateTime EPOCH = Instant.ofEpochSecond(0).atOffset(ZoneOffset.UTC);
+ private static final long FIVE_MINUTES_IN_MICROS = TimeUnit.MINUTES.toMicros(5);
+ private static final long THREE_DAYS_IN_HOURS = TimeUnit.DAYS.toHours(3);
+ private static final long NINETY_DAYS_IN_HOURS = TimeUnit.DAYS.toHours(90);
private static final Pattern DATE = Pattern.compile("\\d\\d\\d\\d-\\d\\d-\\d\\d");
Review Comment:
regex could simpfly to "\\d{4}-\\d{2}-\\d{2}"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #5944: API: Update expression sanitization for relative dates and times
Posted by GitBox <gi...@apache.org>.
rdblue commented on code in PR #5944:
URL: https://github.com/apache/iceberg/pull/5944#discussion_r1002064582
##########
api/src/main/java/org/apache/iceberg/expressions/ExpressionUtil.java:
##########
@@ -33,11 +38,17 @@
public class ExpressionUtil {
private static final Function<Object, Integer> HASH_FUNC =
Transforms.bucket(Integer.MAX_VALUE).bind(Types.StringType.get());
+ private static final OffsetDateTime EPOCH = Instant.ofEpochSecond(0).atOffset(ZoneOffset.UTC);
+ private static final long FIVE_MINUTES_IN_MICROS = TimeUnit.MINUTES.toMicros(5);
+ private static final long THREE_DAYS_IN_HOURS = TimeUnit.DAYS.toHours(3);
+ private static final long NINETY_DAYS_IN_HOURS = TimeUnit.DAYS.toHours(90);
private static final Pattern DATE = Pattern.compile("\\d\\d\\d\\d-\\d\\d-\\d\\d");
Review Comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue merged pull request #5944: API: Update expression sanitization for relative dates and times
Posted by GitBox <gi...@apache.org>.
rdblue merged PR #5944:
URL: https://github.com/apache/iceberg/pull/5944
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] XBaith commented on a diff in pull request #5944: API: Update expression sanitization for relative dates and times
Posted by GitBox <gi...@apache.org>.
XBaith commented on code in PR #5944:
URL: https://github.com/apache/iceberg/pull/5944#discussion_r995530184
##########
api/src/main/java/org/apache/iceberg/expressions/ExpressionUtil.java:
##########
@@ -33,11 +38,17 @@
public class ExpressionUtil {
private static final Function<Object, Integer> HASH_FUNC =
Transforms.bucket(Integer.MAX_VALUE).bind(Types.StringType.get());
+ private static final OffsetDateTime EPOCH = Instant.ofEpochSecond(0).atOffset(ZoneOffset.UTC);
+ private static final long FIVE_MINUTES_IN_MICROS = TimeUnit.MINUTES.toMicros(5);
+ private static final long THREE_DAYS_IN_HOURS = TimeUnit.DAYS.toHours(3);
+ private static final long NINETY_DAYS_IN_HOURS = TimeUnit.DAYS.toHours(90);
private static final Pattern DATE = Pattern.compile("\\d\\d\\d\\d-\\d\\d-\\d\\d");
Review Comment:
regex can be simplified to to "\\d{4}-\\d{2}-\\d{2}", the following expression is the same
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on a diff in pull request #5944: API: Update expression sanitization for relative dates and times
Posted by GitBox <gi...@apache.org>.
rdblue commented on code in PR #5944:
URL: https://github.com/apache/iceberg/pull/5944#discussion_r996581390
##########
api/src/main/java/org/apache/iceberg/expressions/ExpressionUtil.java:
##########
@@ -111,7 +122,15 @@ public static boolean selectsPartitions(
private static class ExpressionSanitizer
extends ExpressionVisitors.ExpressionVisitor<Expression> {
- private static final ExpressionSanitizer INSTANCE = new ExpressionSanitizer();
+ private final long now;
+ private final int today;
+
+ private ExpressionSanitizer() {
+ long nowMillis = System.currentTimeMillis();
+ OffsetDateTime nowDateTime = Instant.ofEpochMilli(nowMillis).atOffset(ZoneOffset.UTC);
+ this.now = nowMillis * 1000;
Review Comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org