You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2019/12/24 10:40:32 UTC
[GitHub] [druid] jadireddi opened a new pull request #9096: confluent schema
registry client to accept config and headers
jadireddi opened a new pull request #9096: confluent schema registry client to accept config and headers
URL: https://github.com/apache/druid/pull/9096
Fixes #8806 .
### Description
Enhancing schema registry client i.e. `SchemaRegistryBasedAvroBytesDecoder` to accept additional config's, header's and able to query schema's from multi schema instances.
1. Changed `SchemaRegistryBasedAvroBytesDecoder` to accept `config` and `header's` as Json properties.
2. Added support to give multi url's for multi schema instances by sending as array: `urls`
3. Upgraded confluent version to `5.2.0`, which is compatible with existing kafka version: `2.2.1`
4. Changed depreciated method `getByID` to `getById`.
This PR has:
- [ x] been self-reviewed.
- [x] added documentation for new or modified features or behaviors.
- [ x] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
- [x ] added unit tests or modified existing tests to cover new code paths.
- [ x] been tested in a test Druid cluster.
##### Key changed/added classes in this PR
* `SchemaRegistryBasedAvroBytesDecoder`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] stale[bot] commented on issue #9096: confluent schema
registry client to accept config and headers
Posted by GitBox <gi...@apache.org>.
stale[bot] commented on issue #9096: confluent schema registry client to accept config and headers
URL: https://github.com/apache/druid/pull/9096#issuecomment-589958879
This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 4 weeks if no further activity occurs. If you think that's incorrect or this pull request should instead be reviewed, please simply write any comment. Even if closed, you can still revive the PR at any time or discuss it on the dev@druid.apache.org list. Thank you for your contributions.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] stale[bot] commented on issue #9096: confluent schema
registry client to accept config and headers
Posted by GitBox <gi...@apache.org>.
stale[bot] commented on issue #9096: confluent schema registry client to accept config and headers
URL: https://github.com/apache/druid/pull/9096#issuecomment-591798248
This pull request/issue is no longer marked as stale.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] clintropolis commented on a change in pull request #9096:
confluent schema registry client to accept config and headers
Posted by GitBox <gi...@apache.org>.
clintropolis commented on a change in pull request #9096: confluent schema registry client to accept config and headers
URL: https://github.com/apache/druid/pull/9096#discussion_r384925713
##########
File path: extensions-core/avro-extensions/pom.xml
##########
@@ -36,7 +36,7 @@
<properties>
<schemarepo.version>0.1.3</schemarepo.version>
- <confluent.version>3.0.1</confluent.version>
+ <confluent.version>5.2.0</confluent.version>
Review comment:
[This is why CI is failing](https://travis-ci.org/apache/druid/jobs/629090910#L2971), could you update the entry for this in [licenses.yaml](https://github.com/apache/druid/blob/master/licenses.yaml#L2930). We use this file to ensure we keep our `LICENSE` information correct to make doing releases easier.
Based on the error message in CI, it looks like maybe the new version pulls in a few additional jars, so you might need to add a few additional entries for `extensions/druid-avro-extensions` (or exclude these jars if they aren't actually needed for operation of the extension)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] stale[bot] commented on issue #9096: confluent schema
registry client to accept config and headers
Posted by GitBox <gi...@apache.org>.
stale[bot] commented on issue #9096: confluent schema registry client to accept config and headers
URL: https://github.com/apache/druid/pull/9096#issuecomment-591798250
This pull request/issue is no longer marked as stale.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] clintropolis commented on a change in pull request #9096:
confluent schema registry client to accept config and headers
Posted by GitBox <gi...@apache.org>.
clintropolis commented on a change in pull request #9096: confluent schema registry client to accept config and headers
URL: https://github.com/apache/druid/pull/9096#discussion_r384926170
##########
File path: extensions-core/avro-extensions/src/main/java/org/apache/druid/data/input/avro/SchemaRegistryBasedAvroBytesDecoder.java
##########
@@ -41,11 +44,18 @@
@JsonCreator
public SchemaRegistryBasedAvroBytesDecoder(
@JsonProperty("url") String url,
- @JsonProperty("capacity") Integer capacity
+ @JsonProperty("capacity") Integer capacity,
+ @JsonProperty("urls") @Nullable List<String> urls,
Review comment:
I think it would probably make sense to mark the `url` parameter as `@Deprecated` in favor of the new `urls` parameter. If you agree, I think we should remove mention of `url` from the avro extension documentation.
Additionally, would you mind adding a JSON serialization/deserialization unit test for this class to make sure the new properties work as expected?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] stale[bot] commented on issue #9096: confluent schema
registry client to accept config and headers
Posted by GitBox <gi...@apache.org>.
stale[bot] commented on issue #9096: confluent schema registry client to accept config and headers
URL: https://github.com/apache/druid/pull/9096#issuecomment-591798247
This pull request/issue is no longer marked as stale.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] stale[bot] commented on issue #9096: confluent schema
registry client to accept config and headers
Posted by GitBox <gi...@apache.org>.
stale[bot] commented on issue #9096: confluent schema registry client to accept config and headers
URL: https://github.com/apache/druid/pull/9096#issuecomment-591798249
This pull request/issue is no longer marked as stale.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org