You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by "zabetak (via GitHub)" <gi...@apache.org> on 2023/08/03 13:17:54 UTC
[GitHub] [hive] zabetak opened a new pull request, #4545: Parse dates from strings ignoring trailing (potentialy) invalid chars
zabetak opened a new pull request, #4545:
URL: https://github.com/apache/hive/pull/4545
Without these changes the parser consumes fully the input string and if the input does not match exactly the expected pattern null is returned.
However there are cases where the input string starts with a completely valid date that matches the pattern but then it is followed by other characters; for example timestamps.
The `DateParser` can handle `2023-08-03T16:02:00` but will return null for `2023-08-03_16:02:00`. Interestingly, it also handles fine `2023-08-03TGARBAGE` but not `2023-08-03_GARBAGE`.
With these changes the `DateParser` can now extract valid dates even if there are trailing invalid chars after the expected pattern.
The changes restore backward compatibility when parting dates since partial parsing used to work before HIVE-20007. At the same time, it makes parsing more uniform since we do not treat 'T' or space (' ') as special cases and simplifies the parsing code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #4545: Parse dates from strings ignoring trailing (potentialy) invalid chars
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4545:
URL: https://github.com/apache/hive/pull/4545#issuecomment-1669883171
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4545)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4545&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4545&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4545&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=CODE_SMELL) [9 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4545&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4545&metric=duplicated_lines_density&view=list) No Duplication information
 The version of Java (11.0.8) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17.
Read more [here](https://docs.sonarcloud.io/appendices/scanner-environment/)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #4545: Parse dates from strings ignoring trailing (potentialy) invalid chars
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4545:
URL: https://github.com/apache/hive/pull/4545#issuecomment-1669340376
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4545)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4545&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4545&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4545&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4545&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4545&metric=duplicated_lines_density&view=list) No Duplication information
 The version of Java (11.0.8) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17.
Read more [here](https://docs.sonarcloud.io/appendices/scanner-environment/)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #4545: HIVE-27586: Parse dates from strings ignoring trailing (potentialy) invalid chars
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4545:
URL: https://github.com/apache/hive/pull/4545#issuecomment-1678483813
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4545)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4545&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4545&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4545&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4545&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4545&metric=duplicated_lines_density&view=list) No Duplication information
 The version of Java (11.0.8) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17.
Read more [here](https://docs.sonarcloud.io/appendices/scanner-environment/)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #4545: Parse dates from strings ignoring trailing (potentialy) invalid chars
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4545:
URL: https://github.com/apache/hive/pull/4545#issuecomment-1664082620
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4545)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4545&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4545&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4545&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4545&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4545&metric=duplicated_lines_density&view=list) No Duplication information
 The version of Java (11.0.8) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17.
Read more [here](https://docs.sonarcloud.io/appendices/scanner-environment/)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zabetak commented on a diff in pull request #4545: HIVE-27586: Parse dates from strings ignoring trailing (potentialy) invalid chars
Posted by "zabetak (via GitHub)" <gi...@apache.org>.
zabetak commented on code in PR #4545:
URL: https://github.com/apache/hive/pull/4545#discussion_r1289747280
##########
common/src/java/org/apache/hadoop/hive/common/type/Date.java:
##########
@@ -175,22 +178,16 @@ public void setTimeInMillis(long epochMilli) {
*/
public static Date valueOf(final String text) {
String s = Objects.requireNonNull(text).trim();
- int idx = s.indexOf(" ");
- if (idx != -1) {
- s = s.substring(0, idx);
- } else {
- idx = s.indexOf('T');
- if (idx != -1) {
- s = s.substring(0, idx);
- }
- }
- LocalDate localDate;
+ ParsePosition pos = new ParsePosition(0);
try {
- localDate = LocalDate.parse(s, PARSE_FORMATTER);
+ TemporalAccessor t = PARSE_FORMATTER.parseUnresolved(s, pos);
Review Comment:
I was thinking that the commit message was enough to understand why `parseUnresolved` is used instead of `parse`.
Usually, I tend to add comments when the code is not easy to understand not to justify why a change was done. I tend to document the "why part" in the commit message/JIRA but this is an entirely personal contribution style.
Anyways more is usually better than less so I will add an appropriate comment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zabetak commented on a diff in pull request #4545: HIVE-27586: Parse dates from strings ignoring trailing (potentialy) invalid chars
Posted by "zabetak (via GitHub)" <gi...@apache.org>.
zabetak commented on code in PR #4545:
URL: https://github.com/apache/hive/pull/4545#discussion_r1289753952
##########
common/src/java/org/apache/hadoop/hive/common/type/Date.java:
##########
@@ -175,22 +178,16 @@ public void setTimeInMillis(long epochMilli) {
*/
public static Date valueOf(final String text) {
String s = Objects.requireNonNull(text).trim();
- int idx = s.indexOf(" ");
- if (idx != -1) {
- s = s.substring(0, idx);
- } else {
- idx = s.indexOf('T');
- if (idx != -1) {
- s = s.substring(0, idx);
- }
- }
- LocalDate localDate;
+ ParsePosition pos = new ParsePosition(0);
try {
- localDate = LocalDate.parse(s, PARSE_FORMATTER);
+ TemporalAccessor t = PARSE_FORMATTER.parseUnresolved(s, pos);
+ if (pos.getErrorIndex() >= 0) {
+ throw new DateTimeParseException("Text could not be parsed to date", s, pos.getErrorIndex());
Review Comment:
Originally, I removed the entire try-catch block to let the `DateTimeException` propagate but this had side-effects since some consumers and tests were expecting the `IllegalArgumentException` to function properly. Then, I decided to not touch the `IllegalArgumentException` as part of this change to minimize such side-effects and unwanted changes in behavior.
I plan to remove the IllegalArgumentException altogether in a follow-up patch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] amansinha100 commented on pull request #4545: HIVE-27586: Parse dates from strings ignoring trailing (potentialy) invalid chars
Posted by "amansinha100 (via GitHub)" <gi...@apache.org>.
amansinha100 commented on PR #4545:
URL: https://github.com/apache/hive/pull/4545#issuecomment-1673502383
Nit: in the commit message 'potentialy' should be spelled 'potentially' .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] sonarcloud[bot] commented on pull request #4545: HIVE-27586: Parse dates from strings ignoring trailing (potentialy) invalid chars
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #4545:
URL: https://github.com/apache/hive/pull/4545#issuecomment-1674125581
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=4545)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4545&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4545&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=4545&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=4545&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4545&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=4545&metric=duplicated_lines_density&view=list) No Duplication information
 The version of Java (11.0.8) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17.
Read more [here](https://docs.sonarcloud.io/appendices/scanner-environment/)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] nrg4878 commented on pull request #4545: HIVE-27586: Parse dates from strings ignoring trailing (potentialy) invalid chars
Posted by "nrg4878 (via GitHub)" <gi...@apache.org>.
nrg4878 commented on PR #4545:
URL: https://github.com/apache/hive/pull/4545#issuecomment-1678930466
Merged to master. Thank you
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] nrg4878 merged pull request #4545: HIVE-27586: Parse dates from strings ignoring trailing (potentialy) invalid chars
Posted by "nrg4878 (via GitHub)" <gi...@apache.org>.
nrg4878 merged PR #4545:
URL: https://github.com/apache/hive/pull/4545
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] amansinha100 commented on a diff in pull request #4545: HIVE-27586: Parse dates from strings ignoring trailing (potentialy) invalid chars
Posted by "amansinha100 (via GitHub)" <gi...@apache.org>.
amansinha100 commented on code in PR #4545:
URL: https://github.com/apache/hive/pull/4545#discussion_r1290354996
##########
common/src/java/org/apache/hadoop/hive/common/type/Date.java:
##########
@@ -175,22 +178,16 @@ public void setTimeInMillis(long epochMilli) {
*/
public static Date valueOf(final String text) {
String s = Objects.requireNonNull(text).trim();
- int idx = s.indexOf(" ");
- if (idx != -1) {
- s = s.substring(0, idx);
- } else {
- idx = s.indexOf('T');
- if (idx != -1) {
- s = s.substring(0, idx);
- }
- }
- LocalDate localDate;
+ ParsePosition pos = new ParsePosition(0);
try {
- localDate = LocalDate.parse(s, PARSE_FORMATTER);
+ TemporalAccessor t = PARSE_FORMATTER.parseUnresolved(s, pos);
Review Comment:
I am good with a brief comment since the word 'Unresolved' in the api makes one pause and think how the resolution is happening.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] amansinha100 commented on a diff in pull request #4545: HIVE-27586: Parse dates from strings ignoring trailing (potentialy) invalid chars
Posted by "amansinha100 (via GitHub)" <gi...@apache.org>.
amansinha100 commented on code in PR #4545:
URL: https://github.com/apache/hive/pull/4545#discussion_r1289623534
##########
common/src/java/org/apache/hadoop/hive/common/type/Date.java:
##########
@@ -175,22 +178,16 @@ public void setTimeInMillis(long epochMilli) {
*/
public static Date valueOf(final String text) {
String s = Objects.requireNonNull(text).trim();
- int idx = s.indexOf(" ");
- if (idx != -1) {
- s = s.substring(0, idx);
- } else {
- idx = s.indexOf('T');
- if (idx != -1) {
- s = s.substring(0, idx);
- }
- }
- LocalDate localDate;
+ ParsePosition pos = new ParsePosition(0);
try {
- localDate = LocalDate.parse(s, PARSE_FORMATTER);
+ TemporalAccessor t = PARSE_FORMATTER.parseUnresolved(s, pos);
Review Comment:
The DateTimeFormatter::parseUnresolved() documentation mentions that it is meant for advanced use cases and that typical applications should use the parse() method. I suppose that since you are accessing the internal state of the TemporalAccessor, it does match such a 'advanced' use case.
It would be useful to add a comment here about why parseUnresolved() is used. Further, it would be useful to include in the comment how the validation is done since this method does not do validation of the actual value (e.g if the month is 15). The validation is deferred to the later call LocalDate.of() method.
##########
common/src/java/org/apache/hadoop/hive/common/type/Date.java:
##########
@@ -175,22 +178,16 @@ public void setTimeInMillis(long epochMilli) {
*/
public static Date valueOf(final String text) {
String s = Objects.requireNonNull(text).trim();
- int idx = s.indexOf(" ");
- if (idx != -1) {
- s = s.substring(0, idx);
- } else {
- idx = s.indexOf('T');
- if (idx != -1) {
- s = s.substring(0, idx);
- }
- }
- LocalDate localDate;
+ ParsePosition pos = new ParsePosition(0);
try {
- localDate = LocalDate.parse(s, PARSE_FORMATTER);
+ TemporalAccessor t = PARSE_FORMATTER.parseUnresolved(s, pos);
+ if (pos.getErrorIndex() >= 0) {
+ throw new DateTimeParseException("Text could not be parsed to date", s, pos.getErrorIndex());
Review Comment:
nit: This exception is thrown but the catch block on line 190 prints a generic error message rather than this specific one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org