You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by GitBox <gi...@apache.org> on 2021/06/29 16:32:12 UTC
[GitHub] [nifi] exceptionfactory opened a new pull request #5194: NIFI-8748 Corrected PutKudu String to java.sql.Date parsing
exceptionfactory opened a new pull request #5194:
URL: https://github.com/apache/nifi/pull/5194
#### Description of PR
NIFI-8748 Corrects `PutKudu` Date field parsing when converting from a `String` Record field value to a `java.sql.Date` for a Kudu `PartialRow`.
The `DataTypeUtils.getDateFormat()` method sets a default time zone of `GMT` on the `SimpleDateFormat` used for parsing, which results in subtracting one day from the input date string. Changing the approach to use a `SimpleDateFormat` with the default system time zone avoids the implicit date conversion and preserves the original date value following the conversion to `java.sql.Date`. The time zone setting is not necessary since the date field does not include precision beyond a day boundary.
Changes include a new unit test method to verify the corrected behavior.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [X] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [X] Does your PR title start with **NIFI-XXXX** where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [X] Has your PR been rebased against the latest commit within the target branch (typically `main`)?
- [X] Is your initial contribution a single, squashed commit? _Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not `squash` or use `--force` when pushing to allow for clean monitoring of changes._
### For code changes:
- [X] Have you ensured that the full suite of tests is executed via `mvn -Pcontrib-check clean install` at the root `nifi` folder?
- [X] Have you written or updated unit tests to verify your changes?
- [X] Have you verified that the full build is successful on JDK 8?
- [X] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main `LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main `NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to .name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for build issues and submit an update to your PR as soon as possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] asfgit closed pull request #5194: NIFI-8748 Corrected PutKudu String to java.sql.Date parsing
Posted by GitBox <gi...@apache.org>.
asfgit closed pull request #5194:
URL: https://github.com/apache/nifi/pull/5194
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] pvillard31 commented on pull request #5194: NIFI-8748 Corrected PutKudu String to java.sql.Date parsing
Posted by GitBox <gi...@apache.org>.
pvillard31 commented on pull request #5194:
URL: https://github.com/apache/nifi/pull/5194#issuecomment-871329761
Merged, thanks all!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a change in pull request #5194: NIFI-8748 Corrected PutKudu String to java.sql.Date parsing
Posted by GitBox <gi...@apache.org>.
exceptionfactory commented on a change in pull request #5194:
URL: https://github.com/apache/nifi/pull/5194#discussion_r660986454
##########
File path: nifi-nar-bundles/nifi-kudu-bundle/nifi-kudu-processors/src/main/java/org/apache/nifi/processors/kudu/AbstractKuduProcessor.java
##########
@@ -412,6 +415,26 @@ protected void buildPartialRow(Schema schema, PartialRow row, Record record, Lis
}
}
+ /**
+ * Get java.sql.Date from Record Field Value with optional parsing when input value is a String
+ *
+ * @param value Record Field Value
+ * @param recordFieldName Record Field Name
+ * @return Date object or null when value is null
+ */
+ private Date getDate(final Object value, final String recordFieldName) {
+ return DataTypeUtils.toDate(value, () -> getDateFormat(), recordFieldName);
+ }
+
+ /**
+ * Get Date Format using Date Record Field default pattern and system time zone to avoid unnecessary conversion
+ *
+ * @return Date Format used to parsing date fields
+ */
+ private DateFormat getDateFormat() {
+ return new SimpleDateFormat(RecordFieldType.DATE.getDefaultFormat());
Review comment:
Thanks for the suggestion @granthenke, that seems like a helpful improvement as opposed to hard-coded default format of `yyyy-MM-dd` from `RecordFieldType.DATE.getDefaultFormat()`. I updated the methods to accept the format from `DataType.getFormat()`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] pvillard31 commented on pull request #5194: NIFI-8748 Corrected PutKudu String to java.sql.Date parsing
Posted by GitBox <gi...@apache.org>.
pvillard31 commented on pull request #5194:
URL: https://github.com/apache/nifi/pull/5194#issuecomment-870756156
@granthenke - can you eyeball this PR? it looks good to me but would appreciate your feedback :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] granthenke commented on pull request #5194: NIFI-8748 Corrected PutKudu String to java.sql.Date parsing
Posted by GitBox <gi...@apache.org>.
granthenke commented on pull request #5194:
URL: https://github.com/apache/nifi/pull/5194#issuecomment-870913886
Looks good to me, clearly an important fix. I just had a question about the use of the default format vs fieldType.getFormat().
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] granthenke commented on pull request #5194: NIFI-8748 Corrected PutKudu String to java.sql.Date parsing
Posted by GitBox <gi...@apache.org>.
granthenke commented on pull request #5194:
URL: https://github.com/apache/nifi/pull/5194#issuecomment-871036319
👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] granthenke commented on a change in pull request #5194: NIFI-8748 Corrected PutKudu String to java.sql.Date parsing
Posted by GitBox <gi...@apache.org>.
granthenke commented on a change in pull request #5194:
URL: https://github.com/apache/nifi/pull/5194#discussion_r660959323
##########
File path: nifi-nar-bundles/nifi-kudu-bundle/nifi-kudu-processors/src/main/java/org/apache/nifi/processors/kudu/AbstractKuduProcessor.java
##########
@@ -412,6 +415,26 @@ protected void buildPartialRow(Schema schema, PartialRow row, Record record, Lis
}
}
+ /**
+ * Get java.sql.Date from Record Field Value with optional parsing when input value is a String
+ *
+ * @param value Record Field Value
+ * @param recordFieldName Record Field Name
+ * @return Date object or null when value is null
+ */
+ private Date getDate(final Object value, final String recordFieldName) {
+ return DataTypeUtils.toDate(value, () -> getDateFormat(), recordFieldName);
+ }
+
+ /**
+ * Get Date Format using Date Record Field default pattern and system time zone to avoid unnecessary conversion
+ *
+ * @return Date Format used to parsing date fields
+ */
+ private DateFormat getDateFormat() {
+ return new SimpleDateFormat(RecordFieldType.DATE.getDefaultFormat());
Review comment:
Should this use `fieldType.getFormat()` similar to the handling for Timestamp/UNIXTIME_MICROS above?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org