You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@pekko.apache.org by "pjfanning (via GitHub)" <gi...@apache.org> on 2024/02/06 10:21:17 UTC
[D] Pekko Connectors 1.1.x dependency upgrades [incubator-pekko-connectors]
GitHub user pjfanning created a discussion: Pekko Connectors 1.1.x dependency upgrades
We were conservative with dependency upgrades in Pekko Connectors 1.0.x to make it easier for users coming from Alpakka to uptake Pekko Connectors.
The aim with Pekko Connectors 1.1.x is to use newer dependencies were feasible.
Some exceptions have come up
* Hbase testing is broken so it makes it difficult to upgrade its connector dependencies - https://github.com/apache/incubator-pekko-connectors/issues/61
* SQS connector doesn't work with latest AWS lib - https://github.com/apache/incubator-pekko-connectors/pull/441
* Latest versions of Spring only support Java 17. We still support Java 8. https://github.com/apache/incubator-pekko-connectors/blob/4223ac20b3eec15fd3a2020b86397c0bcf262056/.scala-steward.conf
* ActiveMQ libs now have Java 11 minimum
* Some test libs like mockito only support Java 11 in their latest releases
* incomplete work to support commons-net 3.10.0 (see https://github.com/apache/incubator-pekko-connectors/pull/171)
* Slick 3.5.0 is not released yet (with its Scala 3 support) - https://github.com/apache/incubator-pekko-connectors/pull/223
* Latest protobuf and related grpc releases cause us issues - https://github.com/apache/incubator-pekko-connectors/pull/277 and others
* couchbase client v3 causes us compike problems - https://github.com/apache/incubator-pekko-connectors/pull/400
* latest kudu jars seem to be restructured in a way that breaks our build
GitHub link: https://github.com/apache/incubator-pekko-connectors/discussions/443
----
This is an automatically sent email for notifications@pekko.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@pekko.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [D] Pekko Connectors 1.1.x dependency upgrades [pekko-connectors]
Posted by "pjfanning (via GitHub)" <gi...@apache.org>.
GitHub user pjfanning added a comment to the discussion: Pekko Connectors 1.1.x dependency upgrades
Have you tried just updating the version of jwt-json-common by adding a dependency to it in your build? I am not aware of any code changes that we had to make to facilitate upgrading the jwt-json-common jar.
GitHub link: https://github.com/apache/pekko-connectors/discussions/443#discussioncomment-9095396
----
This is an automatically sent email for notifications@pekko.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@pekko.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [D] Pekko Connectors 1.1.x dependency upgrades [pekko-connectors]
Posted by "pjfanning (via GitHub)" <gi...@apache.org>.
GitHub user pjfanning added a comment to the discussion: Pekko Connectors 1.1.x dependency upgrades
Sbt has mechanisms to get it not to fail when there are version mismatched. The easiest is too explicit dependencies with a selected version on the transitive dependencies that you have trouble with. You can also use dependencyOverrides.
GitHub link: https://github.com/apache/pekko-connectors/discussions/443#discussioncomment-9097338
----
This is an automatically sent email for notifications@pekko.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@pekko.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [D] Pekko Connectors 1.1.x dependency upgrades [pekko-connectors]
Posted by "btomala (via GitHub)" <gi...@apache.org>.
GitHub user btomala added a comment to the discussion: Pekko Connectors 1.1.x dependency upgrades
Sorry, my bad the dependencies is on ` jwt-json-common` which is used by `jwt-circe`
GitHub link: https://github.com/apache/pekko-connectors/discussions/443#discussioncomment-9095321
----
This is an automatically sent email for notifications@pekko.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@pekko.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [D] Pekko Connectors 1.1.x dependency upgrades [incubator-pekko-connectors]
Posted by "pjfanning (via GitHub)" <gi...@apache.org>.
GitHub user pjfanning edited a discussion: Pekko Connectors 1.1.x dependency upgrades
We were conservative with dependency upgrades in Pekko Connectors 1.0.x to make it easier for users coming from Alpakka to uptake Pekko Connectors.
The aim with Pekko Connectors 1.1.x is to use newer dependencies where feasible.
Some exceptions have come up
* Hbase testing is broken so it makes it difficult to upgrade its connector dependencies - https://github.com/apache/incubator-pekko-connectors/issues/61
* SQS connector doesn't work with latest AWS lib - https://github.com/apache/incubator-pekko-connectors/pull/441
* Latest versions of Spring only support Java 17. We still support Java 8. https://github.com/apache/incubator-pekko-connectors/blob/4223ac20b3eec15fd3a2020b86397c0bcf262056/.scala-steward.conf
* ActiveMQ libs now have Java 11 minimum
* Some test libs like mockito only support Java 11 in their latest releases
* incomplete work to support commons-net 3.10.0 (see https://github.com/apache/incubator-pekko-connectors/pull/171)
* Slick 3.5.0 is not released yet (with its Scala 3 support) - https://github.com/apache/incubator-pekko-connectors/pull/223
* Latest protobuf and related grpc releases cause us issues - https://github.com/apache/incubator-pekko-connectors/pull/277 and others
* couchbase client v3 causes us compike problems - https://github.com/apache/incubator-pekko-connectors/pull/400
* latest kudu jars seem to be restructured in a way that breaks our build
* orientdb 3.2 lib causes us build issues - https://github.com/apache/incubator-pekko-connectors/pull/361
* influxdb upgrade causes test problems - https://github.com/apache/incubator-pekko-connectors/pull/380
* hoverfly upgrade issues
GitHub link: https://github.com/apache/incubator-pekko-connectors/discussions/443
----
This is an automatically sent email for notifications@pekko.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@pekko.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [D] Pekko Connectors 1.1.x dependency upgrades [pekko-connectors]
Posted by "pjfanning (via GitHub)" <gi...@apache.org>.
GitHub user pjfanning edited a comment on the discussion: Pekko Connectors 1.1.x dependency upgrades
Sbt has mechanisms to get it not to fail when there are versions mismatched. The easiest way is to add explicit dependencies with a selected version on the transitive dependencies that you have trouble with. You can also use dependencyOverrides.
GitHub link: https://github.com/apache/pekko-connectors/discussions/443#discussioncomment-9097338
----
This is an automatically sent email for notifications@pekko.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@pekko.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [D] Pekko Connectors 1.1.x dependency upgrades [incubator-pekko-connectors]
Posted by "mdedetrich (via GitHub)" <gi...@apache.org>.
GitHub user mdedetrich added a comment to the discussion: Pekko Connectors 1.1.x dependency upgrades
As I see it, I envisage pekko connectors 1.1.x having multiple milestones as we.slowly enable the latest versions of the sdk's/apis that we wrap over
GitHub link: https://github.com/apache/incubator-pekko-connectors/discussions/443#discussioncomment-8381124
----
This is an automatically sent email for notifications@pekko.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@pekko.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [D] Pekko Connectors 1.1.x dependency upgrades [incubator-pekko-connectors]
Posted by "raboof (via GitHub)" <gi...@apache.org>.
GitHub user raboof added a comment to the discussion: Pekko Connectors 1.1.x dependency upgrades
We should probably make sure those known problems have GitHub issues, and exclude them from scala-steward with a reference to those issues, else we'll keep running into them :).
I think in pekko-connectors we should move a bit faster with breaking changes, both by upgrading to newer major versions of dependencies and by requiring newer Java versions. If that means creating more major versions than 'usual' I'm OK with that. A first step towards that would be #443.
GitHub link: https://github.com/apache/incubator-pekko-connectors/discussions/443#discussioncomment-8506668
----
This is an automatically sent email for notifications@pekko.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@pekko.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [D] Pekko Connectors 1.1.x dependency upgrades [incubator-pekko-connectors]
Posted by "pjfanning (via GitHub)" <gi...@apache.org>.
GitHub user pjfanning edited a discussion: Pekko Connectors 1.1.x dependency upgrades
We were conservative with dependency upgrades in Pekko Connectors 1.0.x to make it easier for users coming from Alpakka to uptake Pekko Connectors.
The aim with Pekko Connectors 1.1.x is to use newer dependencies where feasible.
Some exceptions have come up
* Hbase testing is broken so it makes it difficult to upgrade its connector dependencies - https://github.com/apache/incubator-pekko-connectors/issues/61
* SQS connector doesn't work with latest AWS lib - https://github.com/apache/incubator-pekko-connectors/pull/441
* Latest versions of Spring only support Java 17. We still support Java 8. https://github.com/apache/incubator-pekko-connectors/blob/4223ac20b3eec15fd3a2020b86397c0bcf262056/.scala-steward.conf
* ActiveMQ libs now have Java 11 minimum
* Some test libs like mockito only support Java 11 in their latest releases
* Slick 3.5.0 is not released yet (with its Scala 3 support) - https://github.com/apache/incubator-pekko-connectors/pull/223
* Latest protobuf and related grpc releases cause us issues - https://github.com/apache/incubator-pekko-connectors/pull/277 and others
* couchbase client v3 causes us compike problems - https://github.com/apache/incubator-pekko-connectors/pull/400
* latest kudu jars seem to be restructured in a way that breaks our build
* solr 9 causes us compile issues - https://github.com/apache/incubator-pekko-connectors/pull/379
* hoverfly upgrade issues
* we have started using slf4jv2 in pekko and pekko http 1.1 but some changes need to be made in pekko connectors to support it
GitHub link: https://github.com/apache/incubator-pekko-connectors/discussions/443
----
This is an automatically sent email for notifications@pekko.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@pekko.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [D] Pekko Connectors 1.1.x dependency upgrades [pekko-connectors]
Posted by "btomala (via GitHub)" <gi...@apache.org>.
GitHub user btomala added a comment to the discussion: Pekko Connectors 1.1.x dependency upgrades
I tried but I'm getting version conflicts
```scala
[error] * com.github.jwt-scala:jwt-json-common_2.13:10.0.0 (early-semver) is selected over 7.1.5
[error] +- com.github.jwt-scala:jwt-circe_2.13:10.0.0 (depends on 10.0.0)
[error] +- org.apache.pekko:pekko-connectors-google-common_2.13:1.0.2 (depends on 7.1.5)
```
I managed to downgrade circe in my project.
GitHub link: https://github.com/apache/pekko-connectors/discussions/443#discussioncomment-9096841
----
This is an automatically sent email for notifications@pekko.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@pekko.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [D] Pekko Connectors 1.1.x dependency upgrades [incubator-pekko-connectors]
Posted by "pjfanning (via GitHub)" <gi...@apache.org>.
GitHub user pjfanning edited a discussion: Pekko Connectors 1.1.x dependency upgrades
We were conservative with dependency upgrades in Pekko Connectors 1.0.x to make it easier for users coming from Alpakka to uptake Pekko Connectors.
The aim with Pekko Connectors 1.1.x is to use newer dependencies where feasible.
Some exceptions have come up
* Hbase testing is broken so it makes it difficult to upgrade its connector dependencies - https://github.com/apache/incubator-pekko-connectors/issues/61
* SQS connector doesn't work with latest AWS lib - https://github.com/apache/incubator-pekko-connectors/pull/441
* Latest versions of Spring only support Java 17. We still support Java 8. https://github.com/apache/incubator-pekko-connectors/blob/4223ac20b3eec15fd3a2020b86397c0bcf262056/.scala-steward.conf
* ActiveMQ libs now have Java 11 minimum
* Some test libs like mockito only support Java 11 in their latest releases
* incomplete work to support commons-net 3.10.0 (see https://github.com/apache/incubator-pekko-connectors/pull/171)
* Slick 3.5.0 is not released yet (with its Scala 3 support) - https://github.com/apache/incubator-pekko-connectors/pull/223
* Latest protobuf and related grpc releases cause us issues - https://github.com/apache/incubator-pekko-connectors/pull/277 and others
* couchbase client v3 causes us compike problems - https://github.com/apache/incubator-pekko-connectors/pull/400
* latest kudu jars seem to be restructured in a way that breaks our build
* orientdb 3.2 causes us build issues - https://github.com/apache/incubator-pekko-connectors/pull/361
* hoverfly upgrade issues
GitHub link: https://github.com/apache/incubator-pekko-connectors/discussions/443
----
This is an automatically sent email for notifications@pekko.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@pekko.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [D] Pekko Connectors 1.1.x dependency upgrades [incubator-pekko-connectors]
Posted by "mdedetrich (via GitHub)" <gi...@apache.org>.
GitHub user mdedetrich edited a comment on the discussion: Pekko Connectors 1.1.x dependency upgrades
As I see it, I envisage pekko connectors 1.1.x having multiple milestones as we slowly enable the latest versions of the sdk's/apis that we wrap over
GitHub link: https://github.com/apache/incubator-pekko-connectors/discussions/443#discussioncomment-8381124
----
This is an automatically sent email for notifications@pekko.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@pekko.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [D] Pekko Connectors 1.1.x dependency upgrades [incubator-pekko-connectors]
Posted by "mdedetrich (via GitHub)" <gi...@apache.org>.
GitHub user mdedetrich added a comment to the discussion: Pekko Connectors 1.1.x dependency upgrades
> What is the advantage of milestones over 'normal' releases?
One is kind of semantic, i.e. having multiple releases with only a few modules being changed feels silly.
A more convincing argument would be that I want to explore using SemVer for connectors and that would only make sense for a 1.0.0 release that has all of the latest sdk's along with SBT's SemVer setting being set at which point we would then split into seperate repos which would let us follow SemVer properly since we can release single artifacts.
GitHub link: https://github.com/apache/incubator-pekko-connectors/discussions/443#discussioncomment-8506792
----
This is an automatically sent email for notifications@pekko.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@pekko.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [D] Pekko Connectors 1.1.x dependency upgrades [pekko-connectors]
Posted by "btomala (via GitHub)" <gi...@apache.org>.
GitHub user btomala added a comment to the discussion: Pekko Connectors 1.1.x dependency upgrades
Ohh boy... I didn't expect this to take so much. I'm blocked by `jwt-circe` which is used by the `google-common`, it is already upgraded. I will review the snapshot version and use it or downgrade `circe`. I need to dive into dependencies in that case. Anyway, thanks for the replay.
GitHub link: https://github.com/apache/pekko-connectors/discussions/443#discussioncomment-9095140
----
This is an automatically sent email for notifications@pekko.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@pekko.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [D] Pekko Connectors 1.1.x dependency upgrades [incubator-pekko-connectors]
Posted by "pjfanning (via GitHub)" <gi...@apache.org>.
GitHub user pjfanning edited a discussion: Pekko Connectors 1.1.x dependency upgrades
We were conservative with dependency upgrades in Pekko Connectors 1.0.x to make it easier for users coming from Alpakka to uptake Pekko Connectors.
The aim with Pekko Connectors 1.1.x is to use newer dependencies where feasible.
Some exceptions have come up
* Hbase testing is broken so it makes it difficult to upgrade its connector dependencies - https://github.com/apache/incubator-pekko-connectors/issues/61
* SQS connector doesn't work with latest AWS lib - https://github.com/apache/incubator-pekko-connectors/pull/441
* Latest versions of Spring only support Java 17. We still support Java 8. https://github.com/apache/incubator-pekko-connectors/blob/4223ac20b3eec15fd3a2020b86397c0bcf262056/.scala-steward.conf
* I ran a test locally with the latest jars and using Java 17 and the tests pass
* ActiveMQ libs now have Java 11 minimum
* Some test libs like mockito only support Java 11 in their latest releases
* Latest protobuf and related grpc releases cause us issues - https://github.com/apache/incubator-pekko-connectors/pull/277 and others
* I've tested locally and using pekko-grpc 1.1 snapshots - which have newer protobuf dependencies - means that you can upgrade some of the protobuf linked jars - i.e. pekko-grpc 1.0 compatibility is preventing us from upgrading some protobuf linked jars, like google-cloud-pubsub
* couchbase client v3 causes us compile problems - https://github.com/apache/incubator-pekko-connectors/pull/400
* latest kudu jars seem to be restructured in a way that breaks our build
* solr 9 causes us compile issues - https://github.com/apache/incubator-pekko-connectors/pull/379
* hoverfly upgrade issues
* we have started using slf4jv2 in pekko and pekko http 1.1 but some changes need to be made in pekko connectors to support it
GitHub link: https://github.com/apache/incubator-pekko-connectors/discussions/443
----
This is an automatically sent email for notifications@pekko.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@pekko.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [D] Pekko Connectors 1.1.x dependency upgrades [incubator-pekko-connectors]
Posted by "pjfanning (via GitHub)" <gi...@apache.org>.
GitHub user pjfanning edited a discussion: Pekko Connectors 1.1.x dependency upgrades
We were conservative with dependency upgrades in Pekko Connectors 1.0.x to make it easier for users coming from Alpakka to uptake Pekko Connectors.
The aim with Pekko Connectors 1.1.x is to use newer dependencies were feasible.
Some exceptions have come up
* Hbase testing is broken so it makes it difficult to upgrade its connector dependencies - https://github.com/apache/incubator-pekko-connectors/issues/61
* SQS connector doesn't work with latest AWS lib - https://github.com/apache/incubator-pekko-connectors/pull/441
* Latest versions of Spring only support Java 17. We still support Java 8. https://github.com/apache/incubator-pekko-connectors/blob/4223ac20b3eec15fd3a2020b86397c0bcf262056/.scala-steward.conf
* ActiveMQ libs now have Java 11 minimum
* Some test libs like mockito only support Java 11 in their latest releases
* incomplete work to support commons-net 3.10.0 (see https://github.com/apache/incubator-pekko-connectors/pull/171)
* Slick 3.5.0 is not released yet (with its Scala 3 support) - https://github.com/apache/incubator-pekko-connectors/pull/223
* Latest protobuf and related grpc releases cause us issues - https://github.com/apache/incubator-pekko-connectors/pull/277 and others
* couchbase client v3 causes us compike problems - https://github.com/apache/incubator-pekko-connectors/pull/400
* latest kudu jars seem to be restructured in a way that breaks our build
* orientdb 3.2 causes us build issues - https://github.com/apache/incubator-pekko-connectors/pull/361
* hoverfly upgrade issues
GitHub link: https://github.com/apache/incubator-pekko-connectors/discussions/443
----
This is an automatically sent email for notifications@pekko.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@pekko.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [D] Pekko Connectors 1.1.x dependency upgrades [incubator-pekko-connectors]
Posted by "mdedetrich (via GitHub)" <gi...@apache.org>.
GitHub user mdedetrich added a comment to the discussion: Pekko Connectors 1.1.x dependency upgrades
So as stated in https://github.com/apache/incubator-pekko-connectors/discussions/443#discussioncomment-8381124 what I would envisage is that as we release consecutive milestones as we get more and more connectors updated to their latest respective skd's/libraries.
Then when everything is done we can do a full release of Pekko Connectors 1.1.0 and I think at this point we should seriously consider splitting off the connectors into their own repos. Hopefully by this point https://github.com/pjfanning/sbt-pekko-build will be advanced enough that it will remove most of the boilerplate that we have in sbt builds.
GitHub link: https://github.com/apache/incubator-pekko-connectors/discussions/443#discussioncomment-8506739
----
This is an automatically sent email for notifications@pekko.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@pekko.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [D] Pekko Connectors 1.1.x dependency upgrades [pekko-connectors]
Posted by "btomala (via GitHub)" <gi...@apache.org>.
GitHub user btomala edited a comment on the discussion: Pekko Connectors 1.1.x dependency upgrades
I tried but I'm getting version conflicts
```scala
[error] * com.github.jwt-scala:jwt-json-common_2.13:10.0.0 (early-semver) is selected over 7.1.5
[error] +- com.github.jwt-scala:jwt-circe_2.13:10.0.0 (depends on 10.0.0)
[error] +- org.apache.pekko:pekko-connectors-google-common_2.13:1.0.2 (depends on 7.1.5)
```
I managed to downgrade circe in my project. Thanks
GitHub link: https://github.com/apache/pekko-connectors/discussions/443#discussioncomment-9096841
----
This is an automatically sent email for notifications@pekko.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@pekko.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [D] Pekko Connectors 1.1.x dependency upgrades [incubator-pekko-connectors]
Posted by "pjfanning (via GitHub)" <gi...@apache.org>.
GitHub user pjfanning edited a discussion: Pekko Connectors 1.1.x dependency upgrades
We were conservative with dependency upgrades in Pekko Connectors 1.0.x to make it easier for users coming from Alpakka to uptake Pekko Connectors.
The aim with Pekko Connectors 1.1.x is to use newer dependencies where feasible.
Some exceptions have come up
* Hbase testing is broken so it makes it difficult to upgrade its connector dependencies - https://github.com/apache/incubator-pekko-connectors/issues/61
* SQS connector doesn't work with latest AWS lib - https://github.com/apache/incubator-pekko-connectors/pull/441
* Latest versions of Spring only support Java 17. We still support Java 8. https://github.com/apache/incubator-pekko-connectors/blob/4223ac20b3eec15fd3a2020b86397c0bcf262056/.scala-steward.conf
* ActiveMQ libs now have Java 11 minimum
* Some test libs like mockito only support Java 11 in their latest releases
* incomplete work to support commons-net 3.10.0 (see https://github.com/apache/incubator-pekko-connectors/pull/171)
* Slick 3.5.0 is not released yet (with its Scala 3 support) - https://github.com/apache/incubator-pekko-connectors/pull/223
* Latest protobuf and related grpc releases cause us issues - https://github.com/apache/incubator-pekko-connectors/pull/277 and others
* couchbase client v3 causes us compike problems - https://github.com/apache/incubator-pekko-connectors/pull/400
* latest kudu jars seem to be restructured in a way that breaks our build
* solr 9 causes us compile issues - https://github.com/apache/incubator-pekko-connectors/pull/379
* orientdb 3.2 lib causes us build issues - https://github.com/apache/incubator-pekko-connectors/pull/361
* influxdb upgrade causes test problems - https://github.com/apache/incubator-pekko-connectors/pull/380
* hoverfly upgrade issues
* we have started using slf4jv2 in pekko and pekko http 1.1 but some changes need to be made in pekko connectors to support it
GitHub link: https://github.com/apache/incubator-pekko-connectors/discussions/443
----
This is an automatically sent email for notifications@pekko.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@pekko.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [D] Pekko Connectors 1.1.x dependency upgrades [pekko-connectors]
Posted by "btomala (via GitHub)" <gi...@apache.org>.
GitHub user btomala added a comment to the discussion: Pekko Connectors 1.1.x dependency upgrades
Hi Guys what are the plans for releasing the 1.1.0? This is the final lib blocking us from switching to Pekko. Is there any ticket to work on to help you with the release?
GitHub link: https://github.com/apache/pekko-connectors/discussions/443#discussioncomment-9092625
----
This is an automatically sent email for notifications@pekko.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@pekko.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [D] Pekko Connectors 1.1.x dependency upgrades [pekko-connectors]
Posted by "pjfanning (via GitHub)" <gi...@apache.org>.
GitHub user pjfanning added a comment to the discussion: Pekko Connectors 1.1.x dependency upgrades
I would expect pekko connectors 1.1 in a few months. @btomala which particular lib do you need upgraded? It may be feasible for you to just add an explicit dependency on that jar version that you need.
GitHub link: https://github.com/apache/pekko-connectors/discussions/443#discussioncomment-9093538
----
This is an automatically sent email for notifications@pekko.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@pekko.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [D] Pekko Connectors 1.1.x dependency upgrades [pekko-connectors]
Posted by "pjfanning (via GitHub)" <gi...@apache.org>.
GitHub user pjfanning added a comment to the discussion: Pekko Connectors 1.1.x dependency upgrades
I ran `sbt dependencyTree` on the main branch and the 1.0.x branch. I found no evidence that any module in this repo has a dependency on jwt-circe - direct or transitive.
GitHub link: https://github.com/apache/pekko-connectors/discussions/443#discussioncomment-9095255
----
This is an automatically sent email for notifications@pekko.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@pekko.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [D] Pekko Connectors 1.1.x dependency upgrades [pekko-connectors]
Posted by "mdedetrich (via GitHub)" <gi...@apache.org>.
GitHub user mdedetrich added a comment to the discussion: Pekko Connectors 1.1.x dependency upgrades
Its possible that we will have a milestone earlier than a few months but because of technical (specifically semver) reasons it makes sense to do a full release when every connector module has had its dependencies fully updated because we plan to split out the modules into their own repos at some point.
GitHub link: https://github.com/apache/pekko-connectors/discussions/443#discussioncomment-9095183
----
This is an automatically sent email for notifications@pekko.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@pekko.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [D] Pekko Connectors 1.1.x dependency upgrades [incubator-pekko-connectors]
Posted by "pjfanning (via GitHub)" <gi...@apache.org>.
GitHub user pjfanning edited a discussion: Pekko Connectors 1.1.x dependency upgrades
We were conservative with dependency upgrades in Pekko Connectors 1.0.x to make it easier for users coming from Alpakka to uptake Pekko Connectors.
The aim with Pekko Connectors 1.1.x is to use newer dependencies where feasible.
Some exceptions have come up
* Hbase testing is broken so it makes it difficult to upgrade its connector dependencies - https://github.com/apache/incubator-pekko-connectors/issues/61
* SQS connector doesn't work with latest AWS lib - https://github.com/apache/incubator-pekko-connectors/pull/441
* Latest versions of Spring only support Java 17. We still support Java 8. https://github.com/apache/incubator-pekko-connectors/blob/4223ac20b3eec15fd3a2020b86397c0bcf262056/.scala-steward.conf
* ActiveMQ libs now have Java 11 minimum
* Some test libs like mockito only support Java 11 in their latest releases
* incomplete work to support commons-net 3.10.0 (see https://github.com/apache/incubator-pekko-connectors/pull/171)
* Slick 3.5.0 is not released yet (with its Scala 3 support) - https://github.com/apache/incubator-pekko-connectors/pull/223
* Latest protobuf and related grpc releases cause us issues - https://github.com/apache/incubator-pekko-connectors/pull/277 and others
* couchbase client v3 causes us compike problems - https://github.com/apache/incubator-pekko-connectors/pull/400
* latest kudu jars seem to be restructured in a way that breaks our build
* orientdb 3.2 lib causes us build issues - https://github.com/apache/incubator-pekko-connectors/pull/361
* influxdb upgrade causes test problems - https://github.com/apache/incubator-pekko-connectors/pull/380
* hoverfly upgrade issues
* we have started using slf4jv2 in pekko and pekko http 1.1 but some changes need to be made in pekko connectors to support it
GitHub link: https://github.com/apache/incubator-pekko-connectors/discussions/443
----
This is an automatically sent email for notifications@pekko.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@pekko.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [D] Pekko Connectors 1.1.x dependency upgrades [incubator-pekko-connectors]
Posted by "pjfanning (via GitHub)" <gi...@apache.org>.
GitHub user pjfanning added a comment to the discussion: Pekko Connectors 1.1.x dependency upgrades
I don't know what to do with Pekko Connectors. There is evidence from download stats on Maven and GitHub and email traffic that some connectors are used. These sources also highlight that most of the connectors are barely used. If we do a release, I would prefer if a widely used connector (or more than 1) was the driver.
I would prefer not to do a large number of releases but I don't want to delay the releases too much that we end up with huge changes.
My gut feeling is that the 1.1.0 release should still support Java 8. Maybe the next minor/major release could bump the Java minimum. Pekko 1.1.0 does already bump a few dependencies quite a lot and also upgrading the Java minimum feels like a lot for one minor release.
GitHub link: https://github.com/apache/incubator-pekko-connectors/discussions/443#discussioncomment-8509964
----
This is an automatically sent email for notifications@pekko.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@pekko.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [D] Pekko Connectors 1.1.x dependency upgrades [incubator-pekko-connectors]
Posted by "raboof (via GitHub)" <gi...@apache.org>.
GitHub user raboof added a comment to the discussion: Pekko Connectors 1.1.x dependency upgrades
> what I would envisage is that as we release consecutive milestones as we get more and more connectors updated to their latest respective skd's/libraries
What is the advantage of milestones over 'normal' releases?
> we should seriously consider splitting off the connectors into their own repos
This makes a lot of sense to me.
GitHub link: https://github.com/apache/incubator-pekko-connectors/discussions/443#discussioncomment-8506758
----
This is an automatically sent email for notifications@pekko.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@pekko.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [D] Pekko Connectors 1.1.x dependency upgrades [incubator-pekko-connectors]
Posted by "mdedetrich (via GitHub)" <gi...@apache.org>.
GitHub user mdedetrich edited a comment on the discussion: Pekko Connectors 1.1.x dependency upgrades
> What is the advantage of milestones over 'normal' releases?
One is kind of semantic, i.e. having multiple releases with only a few modules being changed feels silly.
A more convincing argument would be that I want to explore using SemVer for connectors and that would only make sense for a 1.0.0 release that has all of the latest sdk's along with SBT's [`versionScheme`](https://www.scala-sbt.org/1.x/docs/Publishing.html#Version+scheme) setting being set to `SemVer` at which point we would then split into seperate repos which would let us follow SemVer properly since we can release single artifacts.
GitHub link: https://github.com/apache/incubator-pekko-connectors/discussions/443#discussioncomment-8506792
----
This is an automatically sent email for notifications@pekko.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@pekko.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [D] Pekko Connectors 1.1.x dependency upgrades [incubator-pekko-connectors]
Posted by "mdedetrich (via GitHub)" <gi...@apache.org>.
GitHub user mdedetrich added a comment to the discussion: Pekko Connectors 1.1.x dependency upgrades
> I would prefer not to do a large number of releases but I don't want to delay the releases too much that we end up with huge changes.
RIght, we have had this issue before with Pekko 1.1.x and we solved this by agreeing on using milestones. The thing is I don't think we can avoid having a release with huge changes, a lot of the Pekko connectors aside from trivial improvements have a large backlog of changes over time that needed to be done but weren't (i.e. upgrading major/epic versions of the underling wrapper/library) which is what we are experiencing now).
> If we do a release, I would prefer if a widely used connector (or more than 1) was the driver.
Right, I would suggest that this should be the driver from an incremental milestone release until we get to the point where every connector is updated to the latest dependencies. We work on this slowly over time, and when we have enough connectors with one of them being a major one that is fully updated we can then suggest a new milestone release and continue until we are done.
If as you stated by definition that most connectors aren't used but some get decent use then this stepping milestone release is what makes sense for everyone especially considering we have a monorepo now. We don't need to stress ourselves in pushing a 1.1.x quickly and while we may have a decent number of milestones it being a milestone itself is not a problem, its still an official proper ASF "release" that has to be voted on like any other.
> My gut feeling is that the 1.1.0 release should still support Java 8. Maybe the next minor/major release could bump the Java minimum. Pekko 1.1.0 does already bump a few dependencies quite a lot and also upgrading the Java minimum feels like a lot for one minor release.
Agreed, 1.1.0 should still support Java 8. If we have cases where the latest spi/wrapper/underlying library doesn't support Java 8, we should stick with the latest version that supports JDK 1.8 and then release a version 2.0.x of that connector with the latest version that supports JDK 11+ when its been split into its own repo.
GitHub link: https://github.com/apache/incubator-pekko-connectors/discussions/443#discussioncomment-8512785
----
This is an automatically sent email for notifications@pekko.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@pekko.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org
Re: [D] Pekko Connectors 1.1.x dependency upgrades [pekko-connectors]
Posted by "mdedetrich (via GitHub)" <gi...@apache.org>.
GitHub user mdedetrich added a comment to the discussion: Pekko Connectors 1.1.x dependency upgrades
Overriding dependencies in this way may not be enough if the version of Circe is old enough since Circe doesn't guarantee binary compatibility in its releases (and this has caused a lot of pain in the Scala ecosystem).
GitHub link: https://github.com/apache/pekko-connectors/discussions/443#discussioncomment-9097597
----
This is an automatically sent email for notifications@pekko.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@pekko.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org