You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "mattyb149 (via GitHub)" <gi...@apache.org> on 2023/06/14 19:49:47 UTC
[GitHub] [nifi] mattyb149 opened a new pull request, #7380: NIFI-11691: Support VARBINARY and LONGVARBINARY types in PutDatabaseRecord
mattyb149 opened a new pull request, #7380:
URL: https://github.com/apache/nifi/pull/7380
# Summary
[NIFI-11691](https://issues.apache.org/jira/browse/NIFI-11691) This PR adds support for the java.sql.Types VARBINARY and LONGVARBINARY in PutDatabaseRecord. It converts the record field value to `byte[]` and then calls `setBytes()` when setting the parameter value in the PreparedStatement
# Tracking
Please complete the following tracking steps prior to pull request creation.
### Issue Tracking
- [x] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created
### Pull Request Tracking
- [x] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-00000`
- [x] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-00000`
### Pull Request Formatting
- [x] Pull Request based on current revision of the `main` branch
- [x] Pull Request refers to a feature branch with one commit containing changes
# Verification
Please indicate the verification steps performed prior to pull request creation.
### Build
- [ ] Build completed using `mvn clean install -P contrib-check`
- [x] JDK 11
- [ ] JDK 17
### Licensing
- [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html)
- [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files
### Documentation
- [ ] Documentation formatting appears as expected in rendered files
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] exceptionfactory closed pull request #7380: NIFI-11691: Support VARBINARY and LONGVARBINARY types in PutDatabaseRecord
Posted by "exceptionfactory (via GitHub)" <gi...@apache.org>.
exceptionfactory closed pull request #7380: NIFI-11691: Support VARBINARY and LONGVARBINARY types in PutDatabaseRecord
URL: https://github.com/apache/nifi/pull/7380
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] mattyb149 commented on a diff in pull request #7380: NIFI-11691: Support VARBINARY and LONGVARBINARY types in PutDatabaseRecord
Posted by "mattyb149 (via GitHub)" <gi...@apache.org>.
mattyb149 commented on code in PR #7380:
URL: https://github.com/apache/nifi/pull/7380#discussion_r1230112238
##########
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/PutDatabaseRecord.java:
##########
@@ -868,6 +868,33 @@ private void setParameter(PreparedStatement ps, int index, Object value, int fie
throw new IOException("Unable to parse data as CLOB/String " + value, e.getCause());
}
}
+ } else if (sqlType == Types.VARBINARY || sqlType == Types.LONGVARBINARY) {
+ if (fieldSqlType == Types.ARRAY || fieldSqlType == Types.VARCHAR) {
+ if (!(value instanceof byte[])) {
+ if (value == null) {
+ try {
+ ps.setNull(index, Types.BLOB);
+ return;
+ } catch (SQLException e) {
+ throw new IOException("Unable to setNull() on prepared statement" , e);
+ }
+ } else {
+ throw new IOException("Expected VARBINARY/LONGVARBINARY to be of type byte[] but is instead " + value.getClass().getName());
+ }
+ }
+ byte[] byteArray = (byte[]) value;
+ try {
+ ps.setBytes(index, byteArray);
+ } catch (SQLException e) {
+ throw new IOException("Unable to parse binary data " + value, e.getCause());
+ }
+ } else {
+ try {
+ ps.setBytes(index, value.toString().getBytes(StandardCharsets.UTF_8));
Review Comment:
This is just in case the data came in as a String
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #7380: NIFI-11691: Support VARBINARY and LONGVARBINARY types in PutDatabaseRecord
Posted by "exceptionfactory (via GitHub)" <gi...@apache.org>.
exceptionfactory commented on code in PR #7380:
URL: https://github.com/apache/nifi/pull/7380#discussion_r1230099971
##########
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/PutDatabaseRecord.java:
##########
@@ -868,6 +868,33 @@ private void setParameter(PreparedStatement ps, int index, Object value, int fie
throw new IOException("Unable to parse data as CLOB/String " + value, e.getCause());
}
}
+ } else if (sqlType == Types.VARBINARY || sqlType == Types.LONGVARBINARY) {
+ if (fieldSqlType == Types.ARRAY || fieldSqlType == Types.VARCHAR) {
+ if (!(value instanceof byte[])) {
+ if (value == null) {
+ try {
+ ps.setNull(index, Types.BLOB);
+ return;
+ } catch (SQLException e) {
+ throw new IOException("Unable to setNull() on prepared statement" , e);
+ }
+ } else {
+ throw new IOException("Expected VARBINARY/LONGVARBINARY to be of type byte[] but is instead " + value.getClass().getName());
+ }
+ }
+ byte[] byteArray = (byte[]) value;
+ try {
+ ps.setBytes(index, byteArray);
+ } catch (SQLException e) {
+ throw new IOException("Unable to parse binary data " + value, e.getCause());
+ }
+ } else {
+ try {
+ ps.setBytes(index, value.toString().getBytes(StandardCharsets.UTF_8));
+ } catch (SQLException e) {
+ throw new IOException("Unable to parse binary data " + value, e.getCause());
Review Comment:
As noted above, recommend replacing `value` with the length of the bytes.
##########
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/PutDatabaseRecord.java:
##########
@@ -868,6 +868,33 @@ private void setParameter(PreparedStatement ps, int index, Object value, int fie
throw new IOException("Unable to parse data as CLOB/String " + value, e.getCause());
}
}
+ } else if (sqlType == Types.VARBINARY || sqlType == Types.LONGVARBINARY) {
+ if (fieldSqlType == Types.ARRAY || fieldSqlType == Types.VARCHAR) {
+ if (!(value instanceof byte[])) {
+ if (value == null) {
+ try {
+ ps.setNull(index, Types.BLOB);
+ return;
+ } catch (SQLException e) {
+ throw new IOException("Unable to setNull() on prepared statement" , e);
+ }
+ } else {
+ throw new IOException("Expected VARBINARY/LONGVARBINARY to be of type byte[] but is instead " + value.getClass().getName());
+ }
+ }
+ byte[] byteArray = (byte[]) value;
+ try {
+ ps.setBytes(index, byteArray);
+ } catch (SQLException e) {
+ throw new IOException("Unable to parse binary data " + value, e.getCause());
Review Comment:
Given the potential size of these objects, recommend replacing `value` the array length to avoid extremely large log messages.
##########
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/PutDatabaseRecord.java:
##########
@@ -868,6 +868,33 @@ private void setParameter(PreparedStatement ps, int index, Object value, int fie
throw new IOException("Unable to parse data as CLOB/String " + value, e.getCause());
}
}
+ } else if (sqlType == Types.VARBINARY || sqlType == Types.LONGVARBINARY) {
+ if (fieldSqlType == Types.ARRAY || fieldSqlType == Types.VARCHAR) {
+ if (!(value instanceof byte[])) {
+ if (value == null) {
+ try {
+ ps.setNull(index, Types.BLOB);
+ return;
+ } catch (SQLException e) {
+ throw new IOException("Unable to setNull() on prepared statement" , e);
+ }
+ } else {
+ throw new IOException("Expected VARBINARY/LONGVARBINARY to be of type byte[] but is instead " + value.getClass().getName());
+ }
+ }
+ byte[] byteArray = (byte[]) value;
+ try {
+ ps.setBytes(index, byteArray);
+ } catch (SQLException e) {
+ throw new IOException("Unable to parse binary data " + value, e.getCause());
+ }
+ } else {
+ try {
+ ps.setBytes(index, value.toString().getBytes(StandardCharsets.UTF_8));
Review Comment:
It might be worth adding a comment on this line, since `value.toString()` may not provide an expected representation of the underlying object type. Perhaps there should be some additional type checking in this conditional to determine if `value` is an array type or collection type, otherwise it seems like this could produce unexpected results.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org