You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/05/26 15:00:19 UTC
[GitHub] [pulsar] nicoloboschi opened a new pull request, #15807: [improve][connector] JDBC Sinks: support KeyValue schema
nicoloboschi opened a new pull request, #15807:
URL: https://github.com/apache/pulsar/pull/15807
### Motivation
Currently JDBC Sinks only supports Avro or JSON schema.
### Modifications
* Added support for the KeyValue schema.
* All the fields included in the Key schema and in the JSON schema are collected in a HashMap. Then the value for each JDBC column are looked up by the column name (specified in the Sink config).
* If a value is included in the key and in the value, the value one will be taken.
* If a structured field is read, an error will be thrown
* The Sink now is converted from Sink<GenericRecord> to Sink<GenericObject> (which is a compatible change)
* Added tests in the SQLLite module
* Added integration tests in the PostGre sink tester
- [x] `doc-not-needed`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] nicoloboschi commented on pull request #15807: [improve][connector] JDBC Sinks: support KeyValue schema
Posted by GitBox <gi...@apache.org>.
nicoloboschi commented on PR #15807:
URL: https://github.com/apache/pulsar/pull/15807#issuecomment-1140854631
/pulsarbot rerun-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] eolivelli commented on a diff in pull request #15807: [improve][connector] JDBC Sinks: support KeyValue schema
Posted by GitBox <gi...@apache.org>.
eolivelli commented on code in PR #15807:
URL: https://github.com/apache/pulsar/pull/15807#discussion_r884674552
##########
pulsar-io/jdbc/core/src/main/java/org/apache/pulsar/io/jdbc/BaseJdbcAutoSchemaSink.java:
##########
@@ -107,5 +134,56 @@ private static void setColumnValue(PreparedStatement statement, int index, Objec
throw new Exception("Not support value type, need to add it. " + value.getClass());
}
}
+
+ private Object getValueFromJsonNode(final JsonNode fn) {
Review Comment:
Static?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] nicoloboschi commented on pull request #15807: [improve][connector] JDBC Sinks: support KeyValue schema
Posted by GitBox <gi...@apache.org>.
nicoloboschi commented on PR #15807:
URL: https://github.com/apache/pulsar/pull/15807#issuecomment-1139580471
/pulsarbot rerun-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] eolivelli merged pull request #15807: [improve][connector] JDBC Sinks: support KeyValue schema
Posted by GitBox <gi...@apache.org>.
eolivelli merged PR #15807:
URL: https://github.com/apache/pulsar/pull/15807
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org