You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by "janjwerner-confluent (via GitHub)" <gi...@apache.org> on 2023/11/28 17:43:46 UTC
[PR] update confluent's dependencies to common, supported version (druid)
janjwerner-confluent opened a new pull request, #15441:
URL: https://github.com/apache/druid/pull/15441
### Description
Update io.confluent.* dependencies to common, updated version 6.2.12
currently used versions are EOL
This PR has:
- [x ] been self-reviewed.
- [ ] using the [concurrency checklist](https://github.com/apache/druid/blob/master/dev/code-review/concurrency.md) (Remove this item if the PR doesn't have any relation to concurrency.)
- [ ] added documentation for new or modified features or behaviors.
- [ ] a release note entry in the PR description.
- [ ] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
- [ ] added or updated version, license, or notice information in [licenses.yaml](https://github.com/apache/druid/blob/master/dev/license.md)
- [ ] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
- [ ] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for [code coverage](https://github.com/apache/druid/blob/master/dev/code-review/code-coverage.md) is met.
- [ ] added integration tests.
- [ ] been tested in a test Druid cluster.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] update confluent's dependencies to common, supported version (druid)
Posted by "xvrl (via GitHub)" <gi...@apache.org>.
xvrl commented on code in PR #15441:
URL: https://github.com/apache/druid/pull/15441#discussion_r1408398589
##########
extensions-core/avro-extensions/pom.xml:
##########
@@ -35,7 +35,7 @@
<properties>
<schemarepo.version>0.1.3</schemarepo.version>
- <confluent.version>5.5.12</confluent.version>
+ <confluent.version>6.2.12</confluent.version>
Review Comment:
any reason we can't move to a most recent 7.x? If so, then let's add a comment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] update confluent's dependencies to common, supported version (druid)
Posted by "xvrl (via GitHub)" <gi...@apache.org>.
xvrl commented on code in PR #15441:
URL: https://github.com/apache/druid/pull/15441#discussion_r1408399504
##########
extensions-core/avro-extensions/pom.xml:
##########
@@ -35,7 +35,7 @@
<properties>
<schemarepo.version>0.1.3</schemarepo.version>
- <confluent.version>5.5.12</confluent.version>
+ <confluent.version>6.2.12</confluent.version>
Review Comment:
also, maybe we can define the confluent version as a top-level property in the main pom to avoid having to update it in so many places.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] update confluent's dependencies to common, supported version (druid)
Posted by "janjwerner-confluent (via GitHub)" <gi...@apache.org>.
janjwerner-confluent commented on code in PR #15441:
URL: https://github.com/apache/druid/pull/15441#discussion_r1408419407
##########
extensions-core/avro-extensions/pom.xml:
##########
@@ -35,7 +35,7 @@
<properties>
<schemarepo.version>0.1.3</schemarepo.version>
- <confluent.version>5.5.12</confluent.version>
+ <confluent.version>6.2.12</confluent.version>
Review Comment:
I went with closest version that has some life still in it.
Trying to update to the latest available: 7.5.1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] update confluent's dependencies to common, supported version (druid)
Posted by "xvrl (via GitHub)" <gi...@apache.org>.
xvrl commented on code in PR #15441:
URL: https://github.com/apache/druid/pull/15441#discussion_r1408646669
##########
licenses.yaml:
##########
@@ -3184,7 +3184,7 @@ libraries:
---
name: Kafka clients
-version: 5.5.12-ccs
+version: 7.0.11-ccs
Review Comment:
looks like this might need to be updated? I'm seeing license check failures:
```
Error: found 3 missing licenses. These licenses are reported, but missing in the registry
druid_module: protobuf-extensions, groupId: io.confluent, artifactId: kafka-schema-registry-client, version: 6.2.12, license: Apache License version 2.0
druid_module: protobuf-extensions, groupId: io.confluent, artifactId: common-utils, version: 6.2.12, license: Apache License version 2.0
druid_module: protobuf-extensions, groupId: org.apache.kafka, artifactId: kafka-clients, version: 6.2.12-ccs, license: Apache License version 2.0
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] update confluent's dependencies to common, supported version (druid)
Posted by "xvrl (via GitHub)" <gi...@apache.org>.
xvrl commented on code in PR #15441:
URL: https://github.com/apache/druid/pull/15441#discussion_r1408398589
##########
extensions-core/avro-extensions/pom.xml:
##########
@@ -35,7 +35,7 @@
<properties>
<schemarepo.version>0.1.3</schemarepo.version>
- <confluent.version>5.5.12</confluent.version>
+ <confluent.version>6.2.12</confluent.version>
Review Comment:
any reason we can't move to a most recent 7.x?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] update confluent's dependencies to common, supported version (druid)
Posted by "janjwerner-confluent (via GitHub)" <gi...@apache.org>.
janjwerner-confluent commented on PR #15441:
URL: https://github.com/apache/druid/pull/15441#issuecomment-1831028811
version 6.2.x should be supported until June 8, 2024,
in the meantime hopefully one can figure out while tests are failing with OOM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] update confluent's dependencies to common, supported version (druid)
Posted by "janjwerner-confluent (via GitHub)" <gi...@apache.org>.
janjwerner-confluent commented on PR #15441:
URL: https://github.com/apache/druid/pull/15441#issuecomment-1830861842
version 7.5.1 leads to OOM in avro tests:
`[INFO] Running org.apache.druid.data.input.AvroHadoopInputRowParserTest
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.820 s -- in org.apache.druid.data.input.AvroHadoopInputRowParserTest
[INFO] Running org.apache.druid.data.input.avro.InlineSchemaAvroBytesDecoderTest
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.145 s -- in org.apache.druid.data.input.avro.InlineSchemaAvroBytesDecoderTest
[INFO] Running org.apache.druid.data.input.avro.AvroOCFInputFormatTest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.027 s -- in org.apache.druid.data.input.avro.AvroOCFInputFormatTest
[INFO] Running org.apache.druid.data.input.avro.SchemaRegistryBasedAvroBytesDecoderTest
java.lang.OutOfMemoryError: Java heap space
Dumping heap to java_pid13052.hprof ...
Heap dump file created [1628054318 bytes in 2.063 secs]`
Trying more modern, still an older version (7.3.5) before reverting to 6.2.x
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] update confluent's dependencies to common, supported version (druid)
Posted by "janjwerner-confluent (via GitHub)" <gi...@apache.org>.
janjwerner-confluent commented on PR #15441:
URL: https://github.com/apache/druid/pull/15441#issuecomment-1830537591
@xvrl
Could you give it a seal of approval?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] update confluent's dependencies to common, supported version (druid)
Posted by "xvrl (via GitHub)" <gi...@apache.org>.
xvrl commented on code in PR #15441:
URL: https://github.com/apache/druid/pull/15441#discussion_r1408646669
##########
licenses.yaml:
##########
@@ -3184,7 +3184,7 @@ libraries:
---
name: Kafka clients
-version: 5.5.12-ccs
+version: 7.0.11-ccs
Review Comment:
looks like this might need to be updated? I'm seeing the following license check failures:
```
Error: found 3 missing licenses. These licenses are reported, but missing in the registry
druid_module: protobuf-extensions, groupId: io.confluent, artifactId: kafka-schema-registry-client, version: 6.2.12, license: Apache License version 2.0
druid_module: protobuf-extensions, groupId: io.confluent, artifactId: common-utils, version: 6.2.12, license: Apache License version 2.0
druid_module: protobuf-extensions, groupId: org.apache.kafka, artifactId: kafka-clients, version: 6.2.12-ccs, license: Apache License version 2.0
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] update confluent's dependencies to common, supported version (druid)
Posted by "xvrl (via GitHub)" <gi...@apache.org>.
xvrl commented on code in PR #15441:
URL: https://github.com/apache/druid/pull/15441#discussion_r1408399504
##########
extensions-core/avro-extensions/pom.xml:
##########
@@ -35,7 +35,7 @@
<properties>
<schemarepo.version>0.1.3</schemarepo.version>
- <confluent.version>5.5.12</confluent.version>
+ <confluent.version>6.2.12</confluent.version>
Review Comment:
also, maybe we can define the confluent version as a top-level property in the main pom to avoid having to remember to update it in so many places.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] update confluent's dependencies to common, supported version (druid)
Posted by "xvrl (via GitHub)" <gi...@apache.org>.
xvrl merged PR #15441:
URL: https://github.com/apache/druid/pull/15441
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org