You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Johnny Miller <jo...@digitalis.io> on 2020/10/29 12:37:24 UTC
What does the community think of the DataStax 4.x Java driver changes?
Hi Everybody,
We wanted to reach out to the community around the v4 changes in the
DataStax Java driver and gauge people's opinions on some of the changes.
DataStax have done a tremendous job over the years on the Cassandra drivers
and contributing to this community. However, we are currently struggling to
adopt the latest driver due to these changes.
We have been working on a project to upgrade an application from v3 to v4.9
of the driver and have encountered major changes between these versions.
We have observed the latest version of the driver contains many more
DataStax Enterprise (DSE) specific code, and this is not surprising as
DataStax have been generous to build it for the Cassandra community.
From our understanding, the DSE specific code must be included even if you
are unable to use it or require it. For example, in CqlSessionBuilder class
which is the main entry point into the driver, there are APIs relating
directly to DataStax Enterprise non-OSS functionality, their cloud DBaaS
etc.. e.g.
- withApplicationName (
https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/session/SessionBuilder.html#withApplicationName-java.lang.String-
)
- withApplicationVersion (
https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/session/SessionBuilder.html#withApplicationVersion-java.lang.String-
)
- withCloudProxyAddress (
https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/session/SessionBuilder.html#withCloudProxyAddress-java.net.InetSocketAddress-
)
- withCloudProxyAddress (
https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/session/SessionBuilder.html#withCloudSecureConnectBundle-java.io.InputStream-
)
plus more.
All of these are sitting under the com.datastax.oss package - not the
com.datastax.dse package.
Additionally the reference.conf for the default driver settings contains a
large number of DSE specific options:
https://github.com/datastax/java-driver/blob/4.9.0/core/src/main/resources/reference.conf
We would like to have seen this implemented in a subclass of the
CqlSessionBuilder eg. DataStaxCqlSessionBuilder and the conf split into two
separate config files.
Additionally, the structure of the library is such that it is bundling all
of the DSE driver code with the non-DSE driver code eg. graph, search etc.
We would also like to have seen DataStax to have implemented it as separate
libs and use a dependency on an OSS only lib in the datastax specific lib
for the shared functionality.
It would be great to be able to only take in the dependencies and code
needed for Apache Cassandra and not the commercial products around it.
However, the above observations are trivial compared to the two core
features of the driver that seem to have been deprecated and we would like
your opinion.
1 - No more failovers to remote-DC
Previous versions of the driver allowed the driver to failover to a remote
DC (if you wanted) in the event of losing access to the local DC. This is
no longer the case.
See:
https://docs.datastax.com/en/developer/java-driver/4.9/manual/core/load_balancing/#default-policy
What this means is either:
a - re-implement this - not trivial
b - initialise multiple instances of the driver in your JVM - not
recommended by DataStax and we have observed some issues when you do this
c - Address the problem via infra e.g load balancer fronting a local
service connecting to local Cassandra DC along side a passive local service
connecting to the remote Cassandra DC
d - stop the entire applications/Cassandra stack in the DC if Cassandra
becomes unavailable to the applications in this DC for any reason.
Not a trivial change to take on, and in our humble opinion undermines one
of the greatest benefits of Apache Cassandra where your applications "just
keep working". We have seen many situations where this has been one of the
core architecture principles for adopting Cassandra and forms part of the
signed off and agreed operational acceptance testing on platforms.
The latest driver will not be able to retain this architecture and it will
be difficult for certain organisations to adopt and be current with the
latest driver.
https://docs.datastax.com/en/developer/java-driver/4.9/manual/core/load_balancing/
vs
https://docs.datastax.com/en/developer/java-driver/3.9/manual/load_balancing/
2 - No more changing consistency on retries
Although this has been a controversial feature, we know quite a few users
of this feature.
Removing this in v4.9 of the driver essentially breaks those applications
and it is not possible to implement this through a custom retry policy as
the driver RetryDecision is now a String enum (
https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/retry/RetryDecision.html)
vs v3.9 (
https://docs.datastax.com/en/drivers/java/3.9/com/datastax/driver/core/policies/RetryPolicy.RetryDecision.html)
where more complex choices could be bubbled up to the driver around the
consistency to retry on. We looked into how we could simulate the old retry
behaviour using the latest driver and it would require non-trivial code
updates within the client application - particularly if the asynchronous
features are leveraged.
Also, the default settings for the driver has remote connections set to 1 -
https://docs.datastax.com/en/developer/java-driver/4.9/manual/core/pooling/#configuration
We are not quite certain the reason or think of a scenario when this is
utilised as local connections are only possible?
https://docs.datastax.com/en/developer/java-driver/4.9/faq/#where-is-downgrading-consistency-retry-policy
We look after many DataStax / Cassandra clusters and these changes in the
latest driver changes have left us scratching our heads.
We would be interested in hearing your thoughts on these observations.
Thanks,
Johnny
--
Johnny Miller
Co-Founder & CTO
https://digitalis.io <http://digitalis.io/> | Work: +44 20805023721|
Mobile: +352 621159920
--
--
The information contained in this electronic message and any
attachments to this message are intended for the exclusive use of the
addressee(s) and may contain proprietary, confidential or privileged
information. If you are not the intended recipient, you should not
disseminate, distribute or copy this e-mail. Please notify the sender
immediately and destroy all copies of this message and any attachments.
WARNING: Computer viruses can be transmitted via email. The recipient
should check this email and any attachments for the presence of viruses.
The company accepts no liability for any damage caused by any virus
transmitted by this email. www.digitalis.io <http://www.digitalis.io>
Re: What does the community think of the DataStax 4.x Java driver changes?
Posted by Johnny Miller <jo...@digitalis.io>.
Alexandre - great news. I am looking forward to using it!!
Regards,
Johnny
On Tue, 19 Jan 2021 at 18:51, Alexandre Dutra <al...@datastax.com>
wrote:
> Hi,
>
> For those not following the Java driver mailing list, I would like to
> point out that we just released driver 4.10.0.
>
> My message to that mailing list has all the details:
>
>
> https://groups.google.com/a/lists.datastax.com/g/java-driver-user/c/CN2UBLoXLtY/m/JE6sqdF1DQAJ
>
> In short, as promised, in this release we are delivering cross-datacenter
> failover and consistency downgrading retries, alongside bug fixes and other
> new features.
>
> Thanks,
> Alex Dutra
>
> On Thu, Dec 10, 2020 at 1:32 PM Alexandre Dutra <
> alexandre.dutra@datastax.com> wrote:
>
>> Hi,
>>
>> To clarify, partly because of recent discussions in this mailing list,
>> and partly because of constant demand from both the community and
>> customers, we (DataStax) decided recently to re-introduce driver-level
>> cross-dc failover and driver-level downgrading retries in driver 4, to
>> be released in driver 4.10.0.
>>
>> These are equivalent to what DCAwareRoundRobinPolicy and
>> DowngradingConsistencyRetryPolicy used to offer in driver 3.
>>
>> We still believe that these features have better alternatives, e.g.
>> application-level variants thereof. But we don't want to be as
>> prescriptive as we used to be. Hence the 2 tickets mentioned,
>> JAVA-2899 and JAVA-2900:
>>
>> https://datastax-oss.atlassian.net/browse/JAVA-2899
>> https://datastax-oss.atlassian.net/browse/JAVA-2900
>>
>> While we are happy to bring these long-awaited features back to driver
>> 4, we also urge users to carefully consider the alternatives when
>> migrating from driver 3 to driver 4.
>>
>> And speaking of alternatives, we have examples of application-level
>> failover and retries that we encourage users to study:
>>
>> 1) Cross-DC failover:
>>
>> https://github.com/datastax/java-driver/blob/java2899/examples/src/main/java/com/datastax/oss/driver/examples/failover/CrossDatacenterFailover.java
>>
>> (This example is not merged yet, it is part of my work for JAVA-2899,
>> but can be used with any version of driver 4.)
>>
>> 2) Downgrading retries:
>>
>> https://github.com/datastax/java-driver/blob/4.x/examples/src/main/java/com/datastax/oss/driver/examples/retry/DowngradingRetry.java
>>
>> Still on the cross-DC failover topic, we also published a white paper
>> a while ago that takes a different (infrastructure-based) approach:
>>
>>
>> https://www.datastax.com/sites/default/files/content/whitepaper/files/2019-09/Designing-Fault-Tolerant-Applications-DataStax.pdf
>>
>> Thanks,
>>
>> Alex Dutra
>>
>>
>> On Wed, Dec 9, 2020 at 10:18 PM Johnny Miller <jo...@digitalis.io>
>> wrote:
>> >
>> > If you haven’t seen it the failover to remote DCs is being added to
>> 4.10 of the java driver (
>> >
>> https://datastax-oss.atlassian.net/plugins/servlet/mobile?originPath=%2Fbrowse%2FJAVA-2899#issue/JAVA-2899
>> and
>> >
>> https://github.com/datastax/java-driver/commit/f4f6da04fd7871eb0e42933fe368a1e4285c710c)
>> and more powerful API around retries (including change the CL) is being
>> discussed here
>> >
>> https://datastax-oss.atlassian.net/plugins/servlet/mobile?originPath=%2Fbrowse%2FJAVA-2900
>> >
>> > This should hopefully make the driver upgrade easier if your using
>> these.
>> >
>> > On Sat, 31 Oct 2020 at 12:53, Cédrick Lunven <
>> cedrick.lunven@datastax.com> wrote:
>> >>
>> >>
>> >> I wrote this, it might help people.
>> >> https://github.com/DataStax-Examples/java-cassandra-driver-from3x-to4x
>> >>
>> >> Full disclosure I am NOT part of the driver team. I was not
>> particularly happy with new version either: required new parameter
>> 'localDC', missing load balancing policy, new application.conf file, new
>> public interface, immutable things enforcing you to do statement =
>> statement.doSomething() or it is not used, custom codec not there, file
>> with some billions keys some hardly been overridden with Programmatic
>> configuration.
>> >>
>> >> My2c
>> >>
>> >> On Fri, Oct 30, 2020 at 11:09 AM Angelo Polo <la...@gmail.com>
>> wrote:
>> >>>
>> >>> Execution profiles are a brilliant new feature of the 4.x drivers.
>> But configuring them was a challenge. Programmatic configuration offers few
>> advantages over the file-based one because options can still only be
>> specified by name for later injection. When runtime settings are to be
>> applied, one would like to be able to provide an instantiated object for a
>> given property (e.g. load balancer). Instead, one must confect a class with
>> bytecode manipulation, or some other means, and/or push information through
>> the DriverContext later and have to deal with timing and mutability issues.
>> It's easy to find oneself deep in the "internal" tree, overriding
>> components far removed from the original target of the customization.
>> >>>
>> >>> Best,
>> >>> Angelo
>> >>>
>> >>> On Fri, Oct 30, 2020 at 4:18 AM Jonathan Koppenhofer <
>> jon@koppedomain.com> wrote:
>> >>>>
>> >>>> We actually feel the same where we have edge cases where downgrading
>> CL was useful. We did end up writing this in application logic as our code
>> was pretty well abstracted and centralized to do so.
>> >>>>
>> >>>> I will also agree the driver seems overly prescriptive now with
>> limited ability to override. Good for protecting users from themselves, but
>> bad for edge cases.
>> >>>>
>> >>>> Generally, we work with alot of application teams, and most of them
>> are putting off the inevitable of the changes required to implement 4.x. In
>> a reasonably well written moderately complex app, it took one of our best
>> developers 2 weeks make the changes, and we were finding bugs for a couple
>> releases after that. Not positive experience.
>> >>>>
>> >>>> We now are done with the migration, and things are working
>> reasonably well.
>> >>>>
>> >>>>
>> >>>> On Thu, Oct 29, 2020, 9:06 AM Johnny Miller <jo...@digitalis.io>
>> wrote:
>> >>>>>
>> >>>>> Joshua - thanks for the update, I have found the ASF slack channel
>> (#cassandra-cep-drivers-donation) and google doc (
>> https://docs.google.com/document/d/1e0SsZxjeTabzrMv99pCz9zIkkgWjUd4KL5Yp0GFzNnY/edit#).
>> Will be watching it closely.
>> >>>>>
>> >>>>> In terms of the functional changes brought into the driver with 4.x
>> the downgrading CL has always been a controversial feature, but the
>> failover to remote DC being removed - I am curious to understand why?
>> >>>>>
>> >>>>> Thanks,
>> >>>>>
>> >>>>> Johnny
>> >>>>>
>> >>>>> On Thu, 29 Oct 2020 at 13:53, Joshua McKenzie <jm...@apache.org>
>> wrote:
>> >>>>>>
>> >>>>>> That's an immense amount of incredibly useful feedback Johnny.
>> Thanks for taking the time and energy to write all this up.
>> >>>>>>
>> >>>>>> I work with some of the engineers who authored these changes in
>> the driver and have brought this thread to their attention. The authors
>> have offered the driver as a CEP donation to the C* project so we will have
>> one in tree which should give a clear path to fixing some of these API
>> issues as well as the loss of functionality on a major.
>> >>>>>>
>> >>>>>>
>> >>>>>> On Thu, Oct 29, 2020 at 8:37 AM, Johnny Miller <
>> johnny@digitalis.io> wrote:
>> >>>>>>>
>> >>>>>>> Hi Everybody,
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> We wanted to reach out to the community around the v4 changes in
>> the DataStax Java driver and gauge people's opinions on some of the
>> changes. DataStax have done a tremendous job over the years on the
>> Cassandra drivers and contributing to this community. However, we are
>> currently struggling to adopt the latest driver due to these changes.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> We have been working on a project to upgrade an application from
>> v3 to v4.9 of the driver and have encountered major changes between these
>> versions.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> We have observed the latest version of the driver contains many
>> more DataStax Enterprise (DSE) specific code, and this is not surprising as
>> DataStax have been generous to build it for the Cassandra community.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> From our understanding, the DSE specific code must be included
>> even if you are unable to use it or require it. For example, in
>> CqlSessionBuilder class which is the main entry point into the driver,
>> there are APIs relating directly to DataStax Enterprise non-OSS
>> functionality, their cloud DBaaS etc.. e.g.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> - withApplicationName (
>> https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/session/SessionBuilder.html#withApplicationName-java.lang.String-
>> )
>> >>>>>>>
>> >>>>>>> - withApplicationVersion (
>> https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/session/SessionBuilder.html#withApplicationVersion-java.lang.String-
>> )
>> >>>>>>>
>> >>>>>>> - withCloudProxyAddress (
>> https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/session/SessionBuilder.html#withCloudProxyAddress-java.net.InetSocketAddress-
>> )
>> >>>>>>>
>> >>>>>>> - withCloudProxyAddress (
>> https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/session/SessionBuilder.html#withCloudSecureConnectBundle-java.io.InputStream-
>> )
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> plus more.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> All of these are sitting under the com.datastax.oss package - not
>> the com.datastax.dse package.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Additionally the reference.conf for the default driver settings
>> contains a large number of DSE specific options:
>> >>>>>>>
>> >>>>>>>
>> https://github.com/datastax/java-driver/blob/4.9.0/core/src/main/resources/reference.conf
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> We would like to have seen this implemented in a subclass of the
>> CqlSessionBuilder eg. DataStaxCqlSessionBuilder and the conf split into two
>> separate config files.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Additionally, the structure of the library is such that it is
>> bundling all of the DSE driver code with the non-DSE driver code eg. graph,
>> search etc. We would also like to have seen DataStax to have implemented it
>> as separate libs and use a dependency on an OSS only lib in the datastax
>> specific lib for the shared functionality.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> It would be great to be able to only take in the dependencies and
>> code needed for Apache Cassandra and not the commercial products around it.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> However, the above observations are trivial compared to the two
>> core features of the driver that seem to have been deprecated and we would
>> like your opinion.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> 1 - No more failovers to remote-DC
>> >>>>>>>
>> >>>>>>> Previous versions of the driver allowed the driver to failover to
>> a remote DC (if you wanted) in the event of losing access to the local DC.
>> This is no longer the case.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> See:
>> https://docs.datastax.com/en/developer/java-driver/4.9/manual/core/load_balancing/#default-policy
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> What this means is either:
>> >>>>>>>
>> >>>>>>> a - re-implement this - not trivial
>> >>>>>>>
>> >>>>>>> b - initialise multiple instances of the driver in your JVM - not
>> recommended by DataStax and we have observed some issues when you do this
>> >>>>>>>
>> >>>>>>> c - Address the problem via infra e.g load balancer fronting a
>> local service connecting to local Cassandra DC along side a passive local
>> service connecting to the remote Cassandra DC
>> >>>>>>>
>> >>>>>>> d - stop the entire applications/Cassandra stack in the DC if
>> Cassandra becomes unavailable to the applications in this DC for any reason.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Not a trivial change to take on, and in our humble opinion
>> undermines one of the greatest benefits of Apache Cassandra where your
>> applications "just keep working". We have seen many situations where this
>> has been one of the core architecture principles for adopting Cassandra and
>> forms part of the signed off and agreed operational acceptance testing on
>> platforms.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> The latest driver will not be able to retain this architecture
>> and it will be difficult for certain organisations to adopt and be current
>> with the latest driver.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> https://docs.datastax.com/en/developer/java-driver/4.9/manual/core/load_balancing/
>> >>>>>>>
>> >>>>>>> vs
>> >>>>>>>
>> >>>>>>>
>> https://docs.datastax.com/en/developer/java-driver/3.9/manual/load_balancing/
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> 2 - No more changing consistency on retries
>> >>>>>>>
>> >>>>>>> Although this has been a controversial feature, we know quite a
>> few users of this feature.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Removing this in v4.9 of the driver essentially breaks those
>> applications and it is not possible to implement this through a custom
>> retry policy as the driver RetryDecision is now a String enum (
>> https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/retry/RetryDecision.html)
>> vs v3.9 (
>> https://docs.datastax.com/en/drivers/java/3.9/com/datastax/driver/core/policies/RetryPolicy.RetryDecision.html)
>> where more complex choices could be bubbled up to the driver around the
>> consistency to retry on. We looked into how we could simulate the old retry
>> behaviour using the latest driver and it would require non-trivial code
>> updates within the client application - particularly if the asynchronous
>> features are leveraged.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Also, the default settings for the driver has remote connections
>> set to 1 -
>> https://docs.datastax.com/en/developer/java-driver/4.9/manual/core/pooling/#configuration
>> We are not quite certain the reason or think of a scenario when this is
>> utilised as local connections are only possible?
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> https://docs.datastax.com/en/developer/java-driver/4.9/faq/#where-is-downgrading-consistency-retry-policy
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> We look after many DataStax / Cassandra clusters and these
>> changes in the latest driver changes have left us scratching our heads.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> We would be interested in hearing your thoughts on these
>> observations.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Thanks,
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Johnny
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> --
>> >>>>>>>
>> >>>>>>> Johnny Miller
>> >>>>>>>
>> >>>>>>> Co-Founder & CTO
>> >>>>>>>
>> >>>>>>> https://digitalis.io | Work: +44 20805023721| Mobile: +352
>> 621159920
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> --
>> >>>>>>>
>> >>>>>>> The information contained in this electronic message and any
>> attachments to this message are intended for the exclusive use of the
>> addressee(s) and may contain proprietary, confidential or privileged
>> information. If you are not the intended recipient, you should not
>> disseminate, distribute or copy this e-mail. Please notify the sender
>> immediately and destroy all copies of this message and any attachments.
>> WARNING: Computer viruses can be transmitted via email. The recipient
>> should check this email and any attachments for the presence of viruses.
>> The company accepts no liability for any damage caused by any virus
>> transmitted by this email. www.digitalis.io
>> >>>>>>
>> >>>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> --
>> >>>>>
>> >>>>> The information contained in this electronic message and any
>> attachments to this message are intended for the exclusive use of the
>> addressee(s) and may contain proprietary, confidential or privileged
>> information. If you are not the intended recipient, you should not
>> disseminate, distribute or copy this e-mail. Please notify the sender
>> immediately and destroy all copies of this message and any attachments.
>> WARNING: Computer viruses can be transmitted via email. The recipient
>> should check this email and any attachments for the presence of viruses.
>> The company accepts no liability for any damage caused by any virus
>> transmitted by this email. www.digitalis.io
>> >>
>> >>
>> >>
>> >> --
>> >> Cedrick Lunven
>> >> e. cedrick.lunven@datastax.com
>> >> w. www.datastax.com
>> >>
>> > --
>> > Sent from my iPhone, apologies for any typos
>> >
>> >
>> > --
>> >
>> > The information contained in this electronic message and any
>> attachments to this message are intended for the exclusive use of the
>> addressee(s) and may contain proprietary, confidential or privileged
>> information. If you are not the intended recipient, you should not
>> disseminate, distribute or copy this e-mail. Please notify the sender
>> immediately and destroy all copies of this message and any attachments.
>> WARNING: Computer viruses can be transmitted via email. The recipient
>> should check this email and any attachments for the presence of viruses.
>> The company accepts no liability for any damage caused by any virus
>> transmitted by this email. www.digitalis.io
>>
>>
>>
>> --
>> Alexandre Dutra
>> e. alexandre.dutra@datastax.com
>> w. www.datastax.com
>>
>
>
> --
> Alexandre Dutra
> e. alexandre.dutra@datastax.com
> w. www.datastax.com
>
>
--
--
The information contained in this electronic message and any
attachments to this message are intended for the exclusive use of the
addressee(s) and may contain proprietary, confidential or privileged
information. If you are not the intended recipient, you should not
disseminate, distribute or copy this e-mail. Please notify the sender
immediately and destroy all copies of this message and any attachments.
WARNING: Computer viruses can be transmitted via email. The recipient
should check this email and any attachments for the presence of viruses.
The company accepts no liability for any damage caused by any virus
transmitted by this email. www.digitalis.io <http://www.digitalis.io>
Re: What does the community think of the DataStax 4.x Java driver changes?
Posted by Alexandre Dutra <al...@datastax.com>.
Hi,
For those not following the Java driver mailing list, I would like to point
out that we just released driver 4.10.0.
My message to that mailing list has all the details:
https://groups.google.com/a/lists.datastax.com/g/java-driver-user/c/CN2UBLoXLtY/m/JE6sqdF1DQAJ
In short, as promised, in this release we are delivering cross-datacenter
failover and consistency downgrading retries, alongside bug fixes and other
new features.
Thanks,
Alex Dutra
On Thu, Dec 10, 2020 at 1:32 PM Alexandre Dutra <
alexandre.dutra@datastax.com> wrote:
> Hi,
>
> To clarify, partly because of recent discussions in this mailing list,
> and partly because of constant demand from both the community and
> customers, we (DataStax) decided recently to re-introduce driver-level
> cross-dc failover and driver-level downgrading retries in driver 4, to
> be released in driver 4.10.0.
>
> These are equivalent to what DCAwareRoundRobinPolicy and
> DowngradingConsistencyRetryPolicy used to offer in driver 3.
>
> We still believe that these features have better alternatives, e.g.
> application-level variants thereof. But we don't want to be as
> prescriptive as we used to be. Hence the 2 tickets mentioned,
> JAVA-2899 and JAVA-2900:
>
> https://datastax-oss.atlassian.net/browse/JAVA-2899
> https://datastax-oss.atlassian.net/browse/JAVA-2900
>
> While we are happy to bring these long-awaited features back to driver
> 4, we also urge users to carefully consider the alternatives when
> migrating from driver 3 to driver 4.
>
> And speaking of alternatives, we have examples of application-level
> failover and retries that we encourage users to study:
>
> 1) Cross-DC failover:
>
> https://github.com/datastax/java-driver/blob/java2899/examples/src/main/java/com/datastax/oss/driver/examples/failover/CrossDatacenterFailover.java
>
> (This example is not merged yet, it is part of my work for JAVA-2899,
> but can be used with any version of driver 4.)
>
> 2) Downgrading retries:
>
> https://github.com/datastax/java-driver/blob/4.x/examples/src/main/java/com/datastax/oss/driver/examples/retry/DowngradingRetry.java
>
> Still on the cross-DC failover topic, we also published a white paper
> a while ago that takes a different (infrastructure-based) approach:
>
>
> https://www.datastax.com/sites/default/files/content/whitepaper/files/2019-09/Designing-Fault-Tolerant-Applications-DataStax.pdf
>
> Thanks,
>
> Alex Dutra
>
>
> On Wed, Dec 9, 2020 at 10:18 PM Johnny Miller <jo...@digitalis.io> wrote:
> >
> > If you haven’t seen it the failover to remote DCs is being added to
> 4.10 of the java driver (
> >
> https://datastax-oss.atlassian.net/plugins/servlet/mobile?originPath=%2Fbrowse%2FJAVA-2899#issue/JAVA-2899
> and
> >
> https://github.com/datastax/java-driver/commit/f4f6da04fd7871eb0e42933fe368a1e4285c710c)
> and more powerful API around retries (including change the CL) is being
> discussed here
> >
> https://datastax-oss.atlassian.net/plugins/servlet/mobile?originPath=%2Fbrowse%2FJAVA-2900
> >
> > This should hopefully make the driver upgrade easier if your using these.
> >
> > On Sat, 31 Oct 2020 at 12:53, Cédrick Lunven <
> cedrick.lunven@datastax.com> wrote:
> >>
> >>
> >> I wrote this, it might help people.
> >> https://github.com/DataStax-Examples/java-cassandra-driver-from3x-to4x
> >>
> >> Full disclosure I am NOT part of the driver team. I was not
> particularly happy with new version either: required new parameter
> 'localDC', missing load balancing policy, new application.conf file, new
> public interface, immutable things enforcing you to do statement =
> statement.doSomething() or it is not used, custom codec not there, file
> with some billions keys some hardly been overridden with Programmatic
> configuration.
> >>
> >> My2c
> >>
> >> On Fri, Oct 30, 2020 at 11:09 AM Angelo Polo <la...@gmail.com>
> wrote:
> >>>
> >>> Execution profiles are a brilliant new feature of the 4.x drivers. But
> configuring them was a challenge. Programmatic configuration offers few
> advantages over the file-based one because options can still only be
> specified by name for later injection. When runtime settings are to be
> applied, one would like to be able to provide an instantiated object for a
> given property (e.g. load balancer). Instead, one must confect a class with
> bytecode manipulation, or some other means, and/or push information through
> the DriverContext later and have to deal with timing and mutability issues.
> It's easy to find oneself deep in the "internal" tree, overriding
> components far removed from the original target of the customization.
> >>>
> >>> Best,
> >>> Angelo
> >>>
> >>> On Fri, Oct 30, 2020 at 4:18 AM Jonathan Koppenhofer <
> jon@koppedomain.com> wrote:
> >>>>
> >>>> We actually feel the same where we have edge cases where downgrading
> CL was useful. We did end up writing this in application logic as our code
> was pretty well abstracted and centralized to do so.
> >>>>
> >>>> I will also agree the driver seems overly prescriptive now with
> limited ability to override. Good for protecting users from themselves, but
> bad for edge cases.
> >>>>
> >>>> Generally, we work with alot of application teams, and most of them
> are putting off the inevitable of the changes required to implement 4.x. In
> a reasonably well written moderately complex app, it took one of our best
> developers 2 weeks make the changes, and we were finding bugs for a couple
> releases after that. Not positive experience.
> >>>>
> >>>> We now are done with the migration, and things are working reasonably
> well.
> >>>>
> >>>>
> >>>> On Thu, Oct 29, 2020, 9:06 AM Johnny Miller <jo...@digitalis.io>
> wrote:
> >>>>>
> >>>>> Joshua - thanks for the update, I have found the ASF slack channel
> (#cassandra-cep-drivers-donation) and google doc (
> https://docs.google.com/document/d/1e0SsZxjeTabzrMv99pCz9zIkkgWjUd4KL5Yp0GFzNnY/edit#).
> Will be watching it closely.
> >>>>>
> >>>>> In terms of the functional changes brought into the driver with 4.x
> the downgrading CL has always been a controversial feature, but the
> failover to remote DC being removed - I am curious to understand why?
> >>>>>
> >>>>> Thanks,
> >>>>>
> >>>>> Johnny
> >>>>>
> >>>>> On Thu, 29 Oct 2020 at 13:53, Joshua McKenzie <jm...@apache.org>
> wrote:
> >>>>>>
> >>>>>> That's an immense amount of incredibly useful feedback Johnny.
> Thanks for taking the time and energy to write all this up.
> >>>>>>
> >>>>>> I work with some of the engineers who authored these changes in the
> driver and have brought this thread to their attention. The authors have
> offered the driver as a CEP donation to the C* project so we will have one
> in tree which should give a clear path to fixing some of these API issues
> as well as the loss of functionality on a major.
> >>>>>>
> >>>>>>
> >>>>>> On Thu, Oct 29, 2020 at 8:37 AM, Johnny Miller <jo...@digitalis.io>
> wrote:
> >>>>>>>
> >>>>>>> Hi Everybody,
> >>>>>>>
> >>>>>>>
> >>>>>>> We wanted to reach out to the community around the v4 changes in
> the DataStax Java driver and gauge people's opinions on some of the
> changes. DataStax have done a tremendous job over the years on the
> Cassandra drivers and contributing to this community. However, we are
> currently struggling to adopt the latest driver due to these changes.
> >>>>>>>
> >>>>>>>
> >>>>>>> We have been working on a project to upgrade an application from
> v3 to v4.9 of the driver and have encountered major changes between these
> versions.
> >>>>>>>
> >>>>>>>
> >>>>>>> We have observed the latest version of the driver contains many
> more DataStax Enterprise (DSE) specific code, and this is not surprising as
> DataStax have been generous to build it for the Cassandra community.
> >>>>>>>
> >>>>>>>
> >>>>>>> From our understanding, the DSE specific code must be included
> even if you are unable to use it or require it. For example, in
> CqlSessionBuilder class which is the main entry point into the driver,
> there are APIs relating directly to DataStax Enterprise non-OSS
> functionality, their cloud DBaaS etc.. e.g.
> >>>>>>>
> >>>>>>>
> >>>>>>> - withApplicationName (
> https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/session/SessionBuilder.html#withApplicationName-java.lang.String-
> )
> >>>>>>>
> >>>>>>> - withApplicationVersion (
> https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/session/SessionBuilder.html#withApplicationVersion-java.lang.String-
> )
> >>>>>>>
> >>>>>>> - withCloudProxyAddress (
> https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/session/SessionBuilder.html#withCloudProxyAddress-java.net.InetSocketAddress-
> )
> >>>>>>>
> >>>>>>> - withCloudProxyAddress (
> https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/session/SessionBuilder.html#withCloudSecureConnectBundle-java.io.InputStream-
> )
> >>>>>>>
> >>>>>>>
> >>>>>>> plus more.
> >>>>>>>
> >>>>>>>
> >>>>>>> All of these are sitting under the com.datastax.oss package - not
> the com.datastax.dse package.
> >>>>>>>
> >>>>>>>
> >>>>>>> Additionally the reference.conf for the default driver settings
> contains a large number of DSE specific options:
> >>>>>>>
> >>>>>>>
> https://github.com/datastax/java-driver/blob/4.9.0/core/src/main/resources/reference.conf
> >>>>>>>
> >>>>>>>
> >>>>>>> We would like to have seen this implemented in a subclass of the
> CqlSessionBuilder eg. DataStaxCqlSessionBuilder and the conf split into two
> separate config files.
> >>>>>>>
> >>>>>>>
> >>>>>>> Additionally, the structure of the library is such that it is
> bundling all of the DSE driver code with the non-DSE driver code eg. graph,
> search etc. We would also like to have seen DataStax to have implemented it
> as separate libs and use a dependency on an OSS only lib in the datastax
> specific lib for the shared functionality.
> >>>>>>>
> >>>>>>>
> >>>>>>> It would be great to be able to only take in the dependencies and
> code needed for Apache Cassandra and not the commercial products around it.
> >>>>>>>
> >>>>>>>
> >>>>>>> However, the above observations are trivial compared to the two
> core features of the driver that seem to have been deprecated and we would
> like your opinion.
> >>>>>>>
> >>>>>>>
> >>>>>>> 1 - No more failovers to remote-DC
> >>>>>>>
> >>>>>>> Previous versions of the driver allowed the driver to failover to
> a remote DC (if you wanted) in the event of losing access to the local DC.
> This is no longer the case.
> >>>>>>>
> >>>>>>>
> >>>>>>> See:
> https://docs.datastax.com/en/developer/java-driver/4.9/manual/core/load_balancing/#default-policy
> >>>>>>>
> >>>>>>>
> >>>>>>> What this means is either:
> >>>>>>>
> >>>>>>> a - re-implement this - not trivial
> >>>>>>>
> >>>>>>> b - initialise multiple instances of the driver in your JVM - not
> recommended by DataStax and we have observed some issues when you do this
> >>>>>>>
> >>>>>>> c - Address the problem via infra e.g load balancer fronting a
> local service connecting to local Cassandra DC along side a passive local
> service connecting to the remote Cassandra DC
> >>>>>>>
> >>>>>>> d - stop the entire applications/Cassandra stack in the DC if
> Cassandra becomes unavailable to the applications in this DC for any reason.
> >>>>>>>
> >>>>>>>
> >>>>>>> Not a trivial change to take on, and in our humble opinion
> undermines one of the greatest benefits of Apache Cassandra where your
> applications "just keep working". We have seen many situations where this
> has been one of the core architecture principles for adopting Cassandra and
> forms part of the signed off and agreed operational acceptance testing on
> platforms.
> >>>>>>>
> >>>>>>>
> >>>>>>> The latest driver will not be able to retain this architecture and
> it will be difficult for certain organisations to adopt and be current with
> the latest driver.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> https://docs.datastax.com/en/developer/java-driver/4.9/manual/core/load_balancing/
> >>>>>>>
> >>>>>>> vs
> >>>>>>>
> >>>>>>>
> https://docs.datastax.com/en/developer/java-driver/3.9/manual/load_balancing/
> >>>>>>>
> >>>>>>>
> >>>>>>> 2 - No more changing consistency on retries
> >>>>>>>
> >>>>>>> Although this has been a controversial feature, we know quite a
> few users of this feature.
> >>>>>>>
> >>>>>>>
> >>>>>>> Removing this in v4.9 of the driver essentially breaks those
> applications and it is not possible to implement this through a custom
> retry policy as the driver RetryDecision is now a String enum (
> https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/retry/RetryDecision.html)
> vs v3.9 (
> https://docs.datastax.com/en/drivers/java/3.9/com/datastax/driver/core/policies/RetryPolicy.RetryDecision.html)
> where more complex choices could be bubbled up to the driver around the
> consistency to retry on. We looked into how we could simulate the old retry
> behaviour using the latest driver and it would require non-trivial code
> updates within the client application - particularly if the asynchronous
> features are leveraged.
> >>>>>>>
> >>>>>>>
> >>>>>>> Also, the default settings for the driver has remote connections
> set to 1 -
> https://docs.datastax.com/en/developer/java-driver/4.9/manual/core/pooling/#configuration
> We are not quite certain the reason or think of a scenario when this is
> utilised as local connections are only possible?
> >>>>>>>
> >>>>>>>
> >>>>>>>
> https://docs.datastax.com/en/developer/java-driver/4.9/faq/#where-is-downgrading-consistency-retry-policy
> >>>>>>>
> >>>>>>>
> >>>>>>> We look after many DataStax / Cassandra clusters and these changes
> in the latest driver changes have left us scratching our heads.
> >>>>>>>
> >>>>>>>
> >>>>>>> We would be interested in hearing your thoughts on these
> observations.
> >>>>>>>
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>>
> >>>>>>>
> >>>>>>> Johnny
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>>
> >>>>>>> Johnny Miller
> >>>>>>>
> >>>>>>> Co-Founder & CTO
> >>>>>>>
> >>>>>>> https://digitalis.io | Work: +44 20805023721| Mobile: +352
> 621159920
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>>
> >>>>>>> The information contained in this electronic message and any
> attachments to this message are intended for the exclusive use of the
> addressee(s) and may contain proprietary, confidential or privileged
> information. If you are not the intended recipient, you should not
> disseminate, distribute or copy this e-mail. Please notify the sender
> immediately and destroy all copies of this message and any attachments.
> WARNING: Computer viruses can be transmitted via email. The recipient
> should check this email and any attachments for the presence of viruses.
> The company accepts no liability for any damage caused by any virus
> transmitted by this email. www.digitalis.io
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>>
> >>>>> The information contained in this electronic message and any
> attachments to this message are intended for the exclusive use of the
> addressee(s) and may contain proprietary, confidential or privileged
> information. If you are not the intended recipient, you should not
> disseminate, distribute or copy this e-mail. Please notify the sender
> immediately and destroy all copies of this message and any attachments.
> WARNING: Computer viruses can be transmitted via email. The recipient
> should check this email and any attachments for the presence of viruses.
> The company accepts no liability for any damage caused by any virus
> transmitted by this email. www.digitalis.io
> >>
> >>
> >>
> >> --
> >> Cedrick Lunven
> >> e. cedrick.lunven@datastax.com
> >> w. www.datastax.com
> >>
> > --
> > Sent from my iPhone, apologies for any typos
> >
> >
> > --
> >
> > The information contained in this electronic message and any attachments
> to this message are intended for the exclusive use of the addressee(s) and
> may contain proprietary, confidential or privileged information. If you are
> not the intended recipient, you should not disseminate, distribute or copy
> this e-mail. Please notify the sender immediately and destroy all copies of
> this message and any attachments. WARNING: Computer viruses can be
> transmitted via email. The recipient should check this email and any
> attachments for the presence of viruses. The company accepts no liability
> for any damage caused by any virus transmitted by this email.
> www.digitalis.io
>
>
>
> --
> Alexandre Dutra
> e. alexandre.dutra@datastax.com
> w. www.datastax.com
>
--
Alexandre Dutra
e. alexandre.dutra@datastax.com
w. www.datastax.com
Re: What does the community think of the DataStax 4.x Java driver changes?
Posted by Alexandre Dutra <al...@datastax.com>.
Hi,
To clarify, partly because of recent discussions in this mailing list,
and partly because of constant demand from both the community and
customers, we (DataStax) decided recently to re-introduce driver-level
cross-dc failover and driver-level downgrading retries in driver 4, to
be released in driver 4.10.0.
These are equivalent to what DCAwareRoundRobinPolicy and
DowngradingConsistencyRetryPolicy used to offer in driver 3.
We still believe that these features have better alternatives, e.g.
application-level variants thereof. But we don't want to be as
prescriptive as we used to be. Hence the 2 tickets mentioned,
JAVA-2899 and JAVA-2900:
https://datastax-oss.atlassian.net/browse/JAVA-2899
https://datastax-oss.atlassian.net/browse/JAVA-2900
While we are happy to bring these long-awaited features back to driver
4, we also urge users to carefully consider the alternatives when
migrating from driver 3 to driver 4.
And speaking of alternatives, we have examples of application-level
failover and retries that we encourage users to study:
1) Cross-DC failover:
https://github.com/datastax/java-driver/blob/java2899/examples/src/main/java/com/datastax/oss/driver/examples/failover/CrossDatacenterFailover.java
(This example is not merged yet, it is part of my work for JAVA-2899,
but can be used with any version of driver 4.)
2) Downgrading retries:
https://github.com/datastax/java-driver/blob/4.x/examples/src/main/java/com/datastax/oss/driver/examples/retry/DowngradingRetry.java
Still on the cross-DC failover topic, we also published a white paper
a while ago that takes a different (infrastructure-based) approach:
https://www.datastax.com/sites/default/files/content/whitepaper/files/2019-09/Designing-Fault-Tolerant-Applications-DataStax.pdf
Thanks,
Alex Dutra
On Wed, Dec 9, 2020 at 10:18 PM Johnny Miller <jo...@digitalis.io> wrote:
>
> If you haven’t seen it the failover to remote DCs is being added to 4.10 of the java driver (
> https://datastax-oss.atlassian.net/plugins/servlet/mobile?originPath=%2Fbrowse%2FJAVA-2899#issue/JAVA-2899 and
> https://github.com/datastax/java-driver/commit/f4f6da04fd7871eb0e42933fe368a1e4285c710c) and more powerful API around retries (including change the CL) is being discussed here
> https://datastax-oss.atlassian.net/plugins/servlet/mobile?originPath=%2Fbrowse%2FJAVA-2900
>
> This should hopefully make the driver upgrade easier if your using these.
>
> On Sat, 31 Oct 2020 at 12:53, Cédrick Lunven <ce...@datastax.com> wrote:
>>
>>
>> I wrote this, it might help people.
>> https://github.com/DataStax-Examples/java-cassandra-driver-from3x-to4x
>>
>> Full disclosure I am NOT part of the driver team. I was not particularly happy with new version either: required new parameter 'localDC', missing load balancing policy, new application.conf file, new public interface, immutable things enforcing you to do statement = statement.doSomething() or it is not used, custom codec not there, file with some billions keys some hardly been overridden with Programmatic configuration.
>>
>> My2c
>>
>> On Fri, Oct 30, 2020 at 11:09 AM Angelo Polo <la...@gmail.com> wrote:
>>>
>>> Execution profiles are a brilliant new feature of the 4.x drivers. But configuring them was a challenge. Programmatic configuration offers few advantages over the file-based one because options can still only be specified by name for later injection. When runtime settings are to be applied, one would like to be able to provide an instantiated object for a given property (e.g. load balancer). Instead, one must confect a class with bytecode manipulation, or some other means, and/or push information through the DriverContext later and have to deal with timing and mutability issues. It's easy to find oneself deep in the "internal" tree, overriding components far removed from the original target of the customization.
>>>
>>> Best,
>>> Angelo
>>>
>>> On Fri, Oct 30, 2020 at 4:18 AM Jonathan Koppenhofer <jo...@koppedomain.com> wrote:
>>>>
>>>> We actually feel the same where we have edge cases where downgrading CL was useful. We did end up writing this in application logic as our code was pretty well abstracted and centralized to do so.
>>>>
>>>> I will also agree the driver seems overly prescriptive now with limited ability to override. Good for protecting users from themselves, but bad for edge cases.
>>>>
>>>> Generally, we work with alot of application teams, and most of them are putting off the inevitable of the changes required to implement 4.x. In a reasonably well written moderately complex app, it took one of our best developers 2 weeks make the changes, and we were finding bugs for a couple releases after that. Not positive experience.
>>>>
>>>> We now are done with the migration, and things are working reasonably well.
>>>>
>>>>
>>>> On Thu, Oct 29, 2020, 9:06 AM Johnny Miller <jo...@digitalis.io> wrote:
>>>>>
>>>>> Joshua - thanks for the update, I have found the ASF slack channel (#cassandra-cep-drivers-donation) and google doc (https://docs.google.com/document/d/1e0SsZxjeTabzrMv99pCz9zIkkgWjUd4KL5Yp0GFzNnY/edit#). Will be watching it closely.
>>>>>
>>>>> In terms of the functional changes brought into the driver with 4.x the downgrading CL has always been a controversial feature, but the failover to remote DC being removed - I am curious to understand why?
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Johnny
>>>>>
>>>>> On Thu, 29 Oct 2020 at 13:53, Joshua McKenzie <jm...@apache.org> wrote:
>>>>>>
>>>>>> That's an immense amount of incredibly useful feedback Johnny. Thanks for taking the time and energy to write all this up.
>>>>>>
>>>>>> I work with some of the engineers who authored these changes in the driver and have brought this thread to their attention. The authors have offered the driver as a CEP donation to the C* project so we will have one in tree which should give a clear path to fixing some of these API issues as well as the loss of functionality on a major.
>>>>>>
>>>>>>
>>>>>> On Thu, Oct 29, 2020 at 8:37 AM, Johnny Miller <jo...@digitalis.io> wrote:
>>>>>>>
>>>>>>> Hi Everybody,
>>>>>>>
>>>>>>>
>>>>>>> We wanted to reach out to the community around the v4 changes in the DataStax Java driver and gauge people's opinions on some of the changes. DataStax have done a tremendous job over the years on the Cassandra drivers and contributing to this community. However, we are currently struggling to adopt the latest driver due to these changes.
>>>>>>>
>>>>>>>
>>>>>>> We have been working on a project to upgrade an application from v3 to v4.9 of the driver and have encountered major changes between these versions.
>>>>>>>
>>>>>>>
>>>>>>> We have observed the latest version of the driver contains many more DataStax Enterprise (DSE) specific code, and this is not surprising as DataStax have been generous to build it for the Cassandra community.
>>>>>>>
>>>>>>>
>>>>>>> From our understanding, the DSE specific code must be included even if you are unable to use it or require it. For example, in CqlSessionBuilder class which is the main entry point into the driver, there are APIs relating directly to DataStax Enterprise non-OSS functionality, their cloud DBaaS etc.. e.g.
>>>>>>>
>>>>>>>
>>>>>>> - withApplicationName (https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/session/SessionBuilder.html#withApplicationName-java.lang.String-)
>>>>>>>
>>>>>>> - withApplicationVersion (https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/session/SessionBuilder.html#withApplicationVersion-java.lang.String-)
>>>>>>>
>>>>>>> - withCloudProxyAddress (https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/session/SessionBuilder.html#withCloudProxyAddress-java.net.InetSocketAddress-)
>>>>>>>
>>>>>>> - withCloudProxyAddress (https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/session/SessionBuilder.html#withCloudSecureConnectBundle-java.io.InputStream-)
>>>>>>>
>>>>>>>
>>>>>>> plus more.
>>>>>>>
>>>>>>>
>>>>>>> All of these are sitting under the com.datastax.oss package - not the com.datastax.dse package.
>>>>>>>
>>>>>>>
>>>>>>> Additionally the reference.conf for the default driver settings contains a large number of DSE specific options:
>>>>>>>
>>>>>>> https://github.com/datastax/java-driver/blob/4.9.0/core/src/main/resources/reference.conf
>>>>>>>
>>>>>>>
>>>>>>> We would like to have seen this implemented in a subclass of the CqlSessionBuilder eg. DataStaxCqlSessionBuilder and the conf split into two separate config files.
>>>>>>>
>>>>>>>
>>>>>>> Additionally, the structure of the library is such that it is bundling all of the DSE driver code with the non-DSE driver code eg. graph, search etc. We would also like to have seen DataStax to have implemented it as separate libs and use a dependency on an OSS only lib in the datastax specific lib for the shared functionality.
>>>>>>>
>>>>>>>
>>>>>>> It would be great to be able to only take in the dependencies and code needed for Apache Cassandra and not the commercial products around it.
>>>>>>>
>>>>>>>
>>>>>>> However, the above observations are trivial compared to the two core features of the driver that seem to have been deprecated and we would like your opinion.
>>>>>>>
>>>>>>>
>>>>>>> 1 - No more failovers to remote-DC
>>>>>>>
>>>>>>> Previous versions of the driver allowed the driver to failover to a remote DC (if you wanted) in the event of losing access to the local DC. This is no longer the case.
>>>>>>>
>>>>>>>
>>>>>>> See: https://docs.datastax.com/en/developer/java-driver/4.9/manual/core/load_balancing/#default-policy
>>>>>>>
>>>>>>>
>>>>>>> What this means is either:
>>>>>>>
>>>>>>> a - re-implement this - not trivial
>>>>>>>
>>>>>>> b - initialise multiple instances of the driver in your JVM - not recommended by DataStax and we have observed some issues when you do this
>>>>>>>
>>>>>>> c - Address the problem via infra e.g load balancer fronting a local service connecting to local Cassandra DC along side a passive local service connecting to the remote Cassandra DC
>>>>>>>
>>>>>>> d - stop the entire applications/Cassandra stack in the DC if Cassandra becomes unavailable to the applications in this DC for any reason.
>>>>>>>
>>>>>>>
>>>>>>> Not a trivial change to take on, and in our humble opinion undermines one of the greatest benefits of Apache Cassandra where your applications "just keep working". We have seen many situations where this has been one of the core architecture principles for adopting Cassandra and forms part of the signed off and agreed operational acceptance testing on platforms.
>>>>>>>
>>>>>>>
>>>>>>> The latest driver will not be able to retain this architecture and it will be difficult for certain organisations to adopt and be current with the latest driver.
>>>>>>>
>>>>>>>
>>>>>>> https://docs.datastax.com/en/developer/java-driver/4.9/manual/core/load_balancing/
>>>>>>>
>>>>>>> vs
>>>>>>>
>>>>>>> https://docs.datastax.com/en/developer/java-driver/3.9/manual/load_balancing/
>>>>>>>
>>>>>>>
>>>>>>> 2 - No more changing consistency on retries
>>>>>>>
>>>>>>> Although this has been a controversial feature, we know quite a few users of this feature.
>>>>>>>
>>>>>>>
>>>>>>> Removing this in v4.9 of the driver essentially breaks those applications and it is not possible to implement this through a custom retry policy as the driver RetryDecision is now a String enum (https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/retry/RetryDecision.html) vs v3.9 (https://docs.datastax.com/en/drivers/java/3.9/com/datastax/driver/core/policies/RetryPolicy.RetryDecision.html) where more complex choices could be bubbled up to the driver around the consistency to retry on. We looked into how we could simulate the old retry behaviour using the latest driver and it would require non-trivial code updates within the client application - particularly if the asynchronous features are leveraged.
>>>>>>>
>>>>>>>
>>>>>>> Also, the default settings for the driver has remote connections set to 1 - https://docs.datastax.com/en/developer/java-driver/4.9/manual/core/pooling/#configuration We are not quite certain the reason or think of a scenario when this is utilised as local connections are only possible?
>>>>>>>
>>>>>>>
>>>>>>> https://docs.datastax.com/en/developer/java-driver/4.9/faq/#where-is-downgrading-consistency-retry-policy
>>>>>>>
>>>>>>>
>>>>>>> We look after many DataStax / Cassandra clusters and these changes in the latest driver changes have left us scratching our heads.
>>>>>>>
>>>>>>>
>>>>>>> We would be interested in hearing your thoughts on these observations.
>>>>>>>
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>>
>>>>>>> Johnny
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> Johnny Miller
>>>>>>>
>>>>>>> Co-Founder & CTO
>>>>>>>
>>>>>>> https://digitalis.io | Work: +44 20805023721| Mobile: +352 621159920
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.digitalis.io
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.digitalis.io
>>
>>
>>
>> --
>> Cedrick Lunven
>> e. cedrick.lunven@datastax.com
>> w. www.datastax.com
>>
> --
> Sent from my iPhone, apologies for any typos
>
>
> --
>
> The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.digitalis.io
--
Alexandre Dutra
e. alexandre.dutra@datastax.com
w. www.datastax.com
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
For additional commands, e-mail: user-help@cassandra.apache.org
Re: What does the community think of the DataStax 4.x Java driver changes?
Posted by Johnny Miller <jo...@digitalis.io>.
If you haven’t seen it the failover to remote DCs is being added to 4.10
of the java driver (
https://datastax-oss.atlassian.net/plugins/servlet/mobile?originPath=%2Fbrowse%2FJAVA-2899#issue/JAVA-2899
and
https://github.com/datastax/java-driver/commit/f4f6da04fd7871eb0e42933fe368a1e4285c710c)
and more powerful API around retries (including change the CL) is being
discussed here
https://datastax-oss.atlassian.net/plugins/servlet/mobile?originPath=%2Fbrowse%2FJAVA-2900
This should hopefully make the driver upgrade easier if your using these.
On Sat, 31 Oct 2020 at 12:53, Cédrick Lunven <ce...@datastax.com>
wrote:
>
> I wrote this, it might help people.
> https://github.com/DataStax-Examples/java-cassandra-driver-from3x-to4x
>
> *Full disclosure I am NOT part of the driver team. *I was not
> particularly happy with new version either: required new parameter
> 'localDC', missing load balancing policy, new application.conf file, new
> public interface, immutable things enforcing you to do statement =
> statement.doSomething() or it is not used, custom codec not there, file
> with some billions keys some hardly been overridden with Programmatic
> configuration.
>
> My2c
>
> On Fri, Oct 30, 2020 at 11:09 AM Angelo Polo <la...@gmail.com>
> wrote:
>
>> Execution profiles are a brilliant new feature of the 4.x drivers. But
>> configuring them was a challenge. Programmatic configuration offers few
>> advantages over the file-based one because options can still only be
>> specified by name for later injection. When runtime settings are to be
>> applied, one would like to be able to provide an instantiated object for a
>> given property (e.g. load balancer). Instead, one must confect a class with
>> bytecode manipulation, or some other means, and/or push information through
>> the DriverContext later and have to deal with timing and mutability issues.
>> It's easy to find oneself deep in the "internal" tree, overriding
>> components far removed from the original target of the customization.
>>
>> Best,
>> Angelo
>>
>> On Fri, Oct 30, 2020 at 4:18 AM Jonathan Koppenhofer <jo...@koppedomain.com>
>> wrote:
>>
>>> We actually feel the same where we have edge cases where downgrading CL
>>> was useful. We did end up writing this in application logic as our code was
>>> pretty well abstracted and centralized to do so.
>>>
>>> I will also agree the driver seems overly prescriptive now with limited
>>> ability to override. Good for protecting users from themselves, but bad for
>>> edge cases.
>>>
>>> Generally, we work with alot of application teams, and most of them are
>>> putting off the inevitable of the changes required to implement 4.x. In a
>>> reasonably well written moderately complex app, it took one of our best
>>> developers 2 weeks make the changes, and we were finding bugs for a couple
>>> releases after that. Not positive experience.
>>>
>>> We now are done with the migration, and things are working reasonably
>>> well.
>>>
>>>
>>> On Thu, Oct 29, 2020, 9:06 AM Johnny Miller <jo...@digitalis.io> wrote:
>>>
>>>> Joshua - thanks for the update, I have found the ASF slack channel
>>>> (#cassandra-cep-drivers-donation) and google doc (
>>>> https://docs.google.com/document/d/1e0SsZxjeTabzrMv99pCz9zIkkgWjUd4KL5Yp0GFzNnY/edit#).
>>>> Will be watching it closely.
>>>>
>>>> In terms of the functional changes brought into the driver with 4.x the
>>>> downgrading CL has always been a controversial feature, but the failover to
>>>> remote DC being removed - I am curious to understand why?
>>>>
>>>> Thanks,
>>>>
>>>> Johnny
>>>>
>>>> On Thu, 29 Oct 2020 at 13:53, Joshua McKenzie <jm...@apache.org>
>>>> wrote:
>>>>
>>>>> That's an immense amount of incredibly useful feedback Johnny. Thanks
>>>>> for taking the time and energy to write all this up.
>>>>>
>>>>> I work with some of the engineers who authored these changes in the
>>>>> driver and have brought this thread to their attention. The authors have
>>>>> offered the driver as a CEP donation to the C* project so we will have one
>>>>> in tree which should give a clear path to fixing some of these API issues
>>>>> as well as the loss of functionality on a major.
>>>>>
>>>>>
>>>>> On Thu, Oct 29, 2020 at 8:37 AM, Johnny Miller <jo...@digitalis.io>
>>>>> wrote:
>>>>>
>>>>>> Hi Everybody,
>>>>>>
>>>>>>
>>>>>> We wanted to reach out to the community around the v4 changes in the
>>>>>> DataStax Java driver and gauge people's opinions on some of the changes.
>>>>>> DataStax have done a tremendous job over the years on the Cassandra drivers
>>>>>> and contributing to this community. However, we are currently struggling to
>>>>>> adopt the latest driver due to these changes.
>>>>>>
>>>>>>
>>>>>> We have been working on a project to upgrade an application from v3
>>>>>> to v4.9 of the driver and have encountered major changes between these
>>>>>> versions.
>>>>>>
>>>>>>
>>>>>> We have observed the latest version of the driver contains many more
>>>>>> DataStax Enterprise (DSE) specific code, and this is not surprising as
>>>>>> DataStax have been generous to build it for the Cassandra community.
>>>>>>
>>>>>>
>>>>>> From our understanding, the DSE specific code must be included even
>>>>>> if you are unable to use it or require it. For example, in
>>>>>> CqlSessionBuilder class which is the main entry point into the driver,
>>>>>> there are APIs relating directly to DataStax Enterprise non-OSS
>>>>>> functionality, their cloud DBaaS etc.. e.g.
>>>>>>
>>>>>>
>>>>>> - withApplicationName (
>>>>>> https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/session/SessionBuilder.html#withApplicationName-java.lang.String-
>>>>>> )
>>>>>>
>>>>>> - withApplicationVersion (
>>>>>> https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/session/SessionBuilder.html#withApplicationVersion-java.lang.String-
>>>>>> )
>>>>>>
>>>>>> - withCloudProxyAddress (
>>>>>> https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/session/SessionBuilder.html#withCloudProxyAddress-java.net.InetSocketAddress-
>>>>>> )
>>>>>>
>>>>>> - withCloudProxyAddress (
>>>>>> https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/session/SessionBuilder.html#withCloudSecureConnectBundle-java.io.InputStream-
>>>>>> )
>>>>>>
>>>>>>
>>>>>> plus more.
>>>>>>
>>>>>>
>>>>>> All of these are sitting under the com.datastax.oss package - not the
>>>>>> com.datastax.dse package.
>>>>>>
>>>>>>
>>>>>> Additionally the reference.conf for the default driver settings
>>>>>> contains a large number of DSE specific options:
>>>>>>
>>>>>>
>>>>>> https://github.com/datastax/java-driver/blob/4.9.0/core/src/main/resources/reference.conf
>>>>>>
>>>>>>
>>>>>> We would like to have seen this implemented in a subclass of the
>>>>>> CqlSessionBuilder eg. DataStaxCqlSessionBuilder and the conf split into two
>>>>>> separate config files.
>>>>>>
>>>>>>
>>>>>> Additionally, the structure of the library is such that it is
>>>>>> bundling all of the DSE driver code with the non-DSE driver code eg. graph,
>>>>>> search etc. We would also like to have seen DataStax to have implemented it
>>>>>> as separate libs and use a dependency on an OSS only lib in the datastax
>>>>>> specific lib for the shared functionality.
>>>>>>
>>>>>>
>>>>>> It would be great to be able to only take in the dependencies and
>>>>>> code needed for Apache Cassandra and not the commercial products around it.
>>>>>>
>>>>>>
>>>>>> However, the above observations are trivial compared to the two core
>>>>>> features of the driver that seem to have been deprecated and we would like
>>>>>> your opinion.
>>>>>>
>>>>>>
>>>>>> 1 - No more failovers to remote-DC
>>>>>>
>>>>>> Previous versions of the driver allowed the driver to failover to a
>>>>>> remote DC (if you wanted) in the event of losing access to the local DC.
>>>>>> This is no longer the case.
>>>>>>
>>>>>>
>>>>>> See:
>>>>>> https://docs.datastax.com/en/developer/java-driver/4.9/manual/core/load_balancing/#default-policy
>>>>>>
>>>>>>
>>>>>> What this means is either:
>>>>>>
>>>>>> a - re-implement this - not trivial
>>>>>>
>>>>>> b - initialise multiple instances of the driver in your JVM - not
>>>>>> recommended by DataStax and we have observed some issues when you do this
>>>>>>
>>>>>> c - Address the problem via infra e.g load balancer fronting a local
>>>>>> service connecting to local Cassandra DC along side a passive local service
>>>>>> connecting to the remote Cassandra DC
>>>>>>
>>>>>> d - stop the entire applications/Cassandra stack in the DC if
>>>>>> Cassandra becomes unavailable to the applications in this DC for any
>>>>>> reason.
>>>>>>
>>>>>>
>>>>>> Not a trivial change to take on, and in our humble opinion undermines
>>>>>> one of the greatest benefits of Apache Cassandra where your applications
>>>>>> "just keep working". We have seen many situations where this has been one
>>>>>> of the core architecture principles for adopting Cassandra and forms part
>>>>>> of the signed off and agreed operational acceptance testing on platforms.
>>>>>>
>>>>>>
>>>>>> The latest driver will not be able to retain this architecture and it
>>>>>> will be difficult for certain organisations to adopt and be current with
>>>>>> the latest driver.
>>>>>>
>>>>>>
>>>>>>
>>>>>> https://docs.datastax.com/en/developer/java-driver/4.9/manual/core/load_balancing/
>>>>>>
>>>>>> vs
>>>>>>
>>>>>>
>>>>>> https://docs.datastax.com/en/developer/java-driver/3.9/manual/load_balancing/
>>>>>>
>>>>>>
>>>>>> 2 - No more changing consistency on retries
>>>>>>
>>>>>> Although this has been a controversial feature, we know quite a few
>>>>>> users of this feature.
>>>>>>
>>>>>>
>>>>>> Removing this in v4.9 of the driver essentially breaks those
>>>>>> applications and it is not possible to implement this through a custom
>>>>>> retry policy as the driver RetryDecision is now a String enum (
>>>>>> https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/retry/RetryDecision.html)
>>>>>> vs v3.9 (
>>>>>> https://docs.datastax.com/en/drivers/java/3.9/com/datastax/driver/core/policies/RetryPolicy.RetryDecision.html)
>>>>>> where more complex choices could be bubbled up to the driver around the
>>>>>> consistency to retry on. We looked into how we could simulate the old retry
>>>>>> behaviour using the latest driver and it would require non-trivial code
>>>>>> updates within the client application - particularly if the asynchronous
>>>>>> features are leveraged.
>>>>>>
>>>>>>
>>>>>> Also, the default settings for the driver has remote connections set
>>>>>> to 1 -
>>>>>> https://docs.datastax.com/en/developer/java-driver/4.9/manual/core/pooling/#configuration
>>>>>> We are not quite certain the reason or think of a scenario when this is
>>>>>> utilised as local connections are only possible?
>>>>>>
>>>>>>
>>>>>>
>>>>>> https://docs.datastax.com/en/developer/java-driver/4.9/faq/#where-is-downgrading-consistency-retry-policy
>>>>>>
>>>>>>
>>>>>> We look after many DataStax / Cassandra clusters and these changes in
>>>>>> the latest driver changes have left us scratching our heads.
>>>>>>
>>>>>>
>>>>>> We would be interested in hearing your thoughts on these observations.
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>>
>>>>>> Johnny
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>> Johnny Miller
>>>>>>
>>>>>> Co-Founder & CTO
>>>>>>
>>>>>> https://digitalis.io
>>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__digitalis.io_&d=DwMFaQ&c=adz96Xi0w1RHqtPMowiL2g&r=CRDTby7ZKMKOJuR7qj8hGNHMqiMtY_3sbc8FBSZtTdo&m=SbjlNK32WNTSbKmszA9g1RNfF-8qJMzwFQIeKJlvAK8&s=wyOKNUxu-__ODwiGCMpfLzJEvGKVCDs5VJ5YqNu8y1Q&e=> |
>>>>>> Work: +44 20805023721| Mobile: +352 621159920
>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>> The information contained in this electronic message and any
>>>>>> attachments to this message are intended for the exclusive use of the
>>>>>> addressee(s) and may contain proprietary, confidential or privileged
>>>>>> information. If you are not the intended recipient, you should not
>>>>>> disseminate, distribute or copy this e-mail. Please notify the sender
>>>>>> immediately and destroy all copies of this message and any attachments.
>>>>>> WARNING: Computer viruses can be transmitted via email. The recipient
>>>>>> should check this email and any attachments for the presence of viruses.
>>>>>> The company accepts no liability for any damage caused by any virus
>>>>>> transmitted by this email. www.digitalis.io
>>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.digitalis.io&d=DwMFaQ&c=adz96Xi0w1RHqtPMowiL2g&r=CRDTby7ZKMKOJuR7qj8hGNHMqiMtY_3sbc8FBSZtTdo&m=SbjlNK32WNTSbKmszA9g1RNfF-8qJMzwFQIeKJlvAK8&s=KQnoR-TFsGz9Ln6x1k1pIo7JO1fbpjwZ03JwN19ONtQ&e=>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> The information contained in this electronic message and any
>>>> attachments to this message are intended for the exclusive use of the
>>>> addressee(s) and may contain proprietary, confidential or privileged
>>>> information. If you are not the intended recipient, you should not
>>>> disseminate, distribute or copy this e-mail. Please notify the sender
>>>> immediately and destroy all copies of this message and any attachments.
>>>> WARNING: Computer viruses can be transmitted via email. The recipient
>>>> should check this email and any attachments for the presence of viruses.
>>>> The company accepts no liability for any damage caused by any virus
>>>> transmitted by this email. www.digitalis.io
>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.digitalis.io&d=DwMFaQ&c=adz96Xi0w1RHqtPMowiL2g&r=CRDTby7ZKMKOJuR7qj8hGNHMqiMtY_3sbc8FBSZtTdo&m=SbjlNK32WNTSbKmszA9g1RNfF-8qJMzwFQIeKJlvAK8&s=KQnoR-TFsGz9Ln6x1k1pIo7JO1fbpjwZ03JwN19ONtQ&e=>
>>>>
>>>
>
> --
> Cedrick Lunven
> e. cedrick.lunven@datastax.com
> w. www.datastax.com
>
> --
Sent from my iPhone, apologies for any typos
--
--
The information contained in this electronic message and any
attachments to this message are intended for the exclusive use of the
addressee(s) and may contain proprietary, confidential or privileged
information. If you are not the intended recipient, you should not
disseminate, distribute or copy this e-mail. Please notify the sender
immediately and destroy all copies of this message and any attachments.
WARNING: Computer viruses can be transmitted via email. The recipient
should check this email and any attachments for the presence of viruses.
The company accepts no liability for any damage caused by any virus
transmitted by this email. www.digitalis.io <http://www.digitalis.io>
Re: What does the community think of the DataStax 4.x Java driver changes?
Posted by Cédrick Lunven <ce...@datastax.com>.
I wrote this, it might help people.
https://github.com/DataStax-Examples/java-cassandra-driver-from3x-to4x
*Full disclosure I am NOT part of the driver team. *I was not particularly
happy with new version either: required new parameter 'localDC', missing
load balancing policy, new application.conf file, new public interface,
immutable things enforcing you to do statement = statement.doSomething() or
it is not used, custom codec not there, file with some billions keys some
hardly been overridden with Programmatic configuration.
My2c
On Fri, Oct 30, 2020 at 11:09 AM Angelo Polo <la...@gmail.com>
wrote:
> Execution profiles are a brilliant new feature of the 4.x drivers. But
> configuring them was a challenge. Programmatic configuration offers few
> advantages over the file-based one because options can still only be
> specified by name for later injection. When runtime settings are to be
> applied, one would like to be able to provide an instantiated object for a
> given property (e.g. load balancer). Instead, one must confect a class with
> bytecode manipulation, or some other means, and/or push information through
> the DriverContext later and have to deal with timing and mutability issues.
> It's easy to find oneself deep in the "internal" tree, overriding
> components far removed from the original target of the customization.
>
> Best,
> Angelo
>
> On Fri, Oct 30, 2020 at 4:18 AM Jonathan Koppenhofer <jo...@koppedomain.com>
> wrote:
>
>> We actually feel the same where we have edge cases where downgrading CL
>> was useful. We did end up writing this in application logic as our code was
>> pretty well abstracted and centralized to do so.
>>
>> I will also agree the driver seems overly prescriptive now with limited
>> ability to override. Good for protecting users from themselves, but bad for
>> edge cases.
>>
>> Generally, we work with alot of application teams, and most of them are
>> putting off the inevitable of the changes required to implement 4.x. In a
>> reasonably well written moderately complex app, it took one of our best
>> developers 2 weeks make the changes, and we were finding bugs for a couple
>> releases after that. Not positive experience.
>>
>> We now are done with the migration, and things are working reasonably
>> well.
>>
>>
>> On Thu, Oct 29, 2020, 9:06 AM Johnny Miller <jo...@digitalis.io> wrote:
>>
>>> Joshua - thanks for the update, I have found the ASF slack channel
>>> (#cassandra-cep-drivers-donation) and google doc (
>>> https://docs.google.com/document/d/1e0SsZxjeTabzrMv99pCz9zIkkgWjUd4KL5Yp0GFzNnY/edit#).
>>> Will be watching it closely.
>>>
>>> In terms of the functional changes brought into the driver with 4.x the
>>> downgrading CL has always been a controversial feature, but the failover to
>>> remote DC being removed - I am curious to understand why?
>>>
>>> Thanks,
>>>
>>> Johnny
>>>
>>> On Thu, 29 Oct 2020 at 13:53, Joshua McKenzie <jm...@apache.org>
>>> wrote:
>>>
>>>> That's an immense amount of incredibly useful feedback Johnny. Thanks
>>>> for taking the time and energy to write all this up.
>>>>
>>>> I work with some of the engineers who authored these changes in the
>>>> driver and have brought this thread to their attention. The authors have
>>>> offered the driver as a CEP donation to the C* project so we will have one
>>>> in tree which should give a clear path to fixing some of these API issues
>>>> as well as the loss of functionality on a major.
>>>>
>>>>
>>>> On Thu, Oct 29, 2020 at 8:37 AM, Johnny Miller <jo...@digitalis.io>
>>>> wrote:
>>>>
>>>>> Hi Everybody,
>>>>>
>>>>>
>>>>> We wanted to reach out to the community around the v4 changes in the
>>>>> DataStax Java driver and gauge people's opinions on some of the changes.
>>>>> DataStax have done a tremendous job over the years on the Cassandra drivers
>>>>> and contributing to this community. However, we are currently struggling to
>>>>> adopt the latest driver due to these changes.
>>>>>
>>>>>
>>>>> We have been working on a project to upgrade an application from v3 to
>>>>> v4.9 of the driver and have encountered major changes between these
>>>>> versions.
>>>>>
>>>>>
>>>>> We have observed the latest version of the driver contains many more
>>>>> DataStax Enterprise (DSE) specific code, and this is not surprising as
>>>>> DataStax have been generous to build it for the Cassandra community.
>>>>>
>>>>>
>>>>> From our understanding, the DSE specific code must be included even if
>>>>> you are unable to use it or require it. For example, in CqlSessionBuilder
>>>>> class which is the main entry point into the driver, there are APIs
>>>>> relating directly to DataStax Enterprise non-OSS functionality, their cloud
>>>>> DBaaS etc.. e.g.
>>>>>
>>>>>
>>>>> - withApplicationName (
>>>>> https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/session/SessionBuilder.html#withApplicationName-java.lang.String-
>>>>> )
>>>>>
>>>>> - withApplicationVersion (
>>>>> https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/session/SessionBuilder.html#withApplicationVersion-java.lang.String-
>>>>> )
>>>>>
>>>>> - withCloudProxyAddress (
>>>>> https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/session/SessionBuilder.html#withCloudProxyAddress-java.net.InetSocketAddress-
>>>>> )
>>>>>
>>>>> - withCloudProxyAddress (
>>>>> https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/session/SessionBuilder.html#withCloudSecureConnectBundle-java.io.InputStream-
>>>>> )
>>>>>
>>>>>
>>>>> plus more.
>>>>>
>>>>>
>>>>> All of these are sitting under the com.datastax.oss package - not the
>>>>> com.datastax.dse package.
>>>>>
>>>>>
>>>>> Additionally the reference.conf for the default driver settings
>>>>> contains a large number of DSE specific options:
>>>>>
>>>>>
>>>>> https://github.com/datastax/java-driver/blob/4.9.0/core/src/main/resources/reference.conf
>>>>>
>>>>>
>>>>> We would like to have seen this implemented in a subclass of the
>>>>> CqlSessionBuilder eg. DataStaxCqlSessionBuilder and the conf split into two
>>>>> separate config files.
>>>>>
>>>>>
>>>>> Additionally, the structure of the library is such that it is bundling
>>>>> all of the DSE driver code with the non-DSE driver code eg. graph, search
>>>>> etc. We would also like to have seen DataStax to have implemented it as
>>>>> separate libs and use a dependency on an OSS only lib in the datastax
>>>>> specific lib for the shared functionality.
>>>>>
>>>>>
>>>>> It would be great to be able to only take in the dependencies and code
>>>>> needed for Apache Cassandra and not the commercial products around it.
>>>>>
>>>>>
>>>>> However, the above observations are trivial compared to the two core
>>>>> features of the driver that seem to have been deprecated and we would like
>>>>> your opinion.
>>>>>
>>>>>
>>>>> 1 - No more failovers to remote-DC
>>>>>
>>>>> Previous versions of the driver allowed the driver to failover to a
>>>>> remote DC (if you wanted) in the event of losing access to the local DC.
>>>>> This is no longer the case.
>>>>>
>>>>>
>>>>> See:
>>>>> https://docs.datastax.com/en/developer/java-driver/4.9/manual/core/load_balancing/#default-policy
>>>>>
>>>>>
>>>>> What this means is either:
>>>>>
>>>>> a - re-implement this - not trivial
>>>>>
>>>>> b - initialise multiple instances of the driver in your JVM - not
>>>>> recommended by DataStax and we have observed some issues when you do this
>>>>>
>>>>> c - Address the problem via infra e.g load balancer fronting a local
>>>>> service connecting to local Cassandra DC along side a passive local service
>>>>> connecting to the remote Cassandra DC
>>>>>
>>>>> d - stop the entire applications/Cassandra stack in the DC if
>>>>> Cassandra becomes unavailable to the applications in this DC for any
>>>>> reason.
>>>>>
>>>>>
>>>>> Not a trivial change to take on, and in our humble opinion undermines
>>>>> one of the greatest benefits of Apache Cassandra where your applications
>>>>> "just keep working". We have seen many situations where this has been one
>>>>> of the core architecture principles for adopting Cassandra and forms part
>>>>> of the signed off and agreed operational acceptance testing on platforms.
>>>>>
>>>>>
>>>>> The latest driver will not be able to retain this architecture and it
>>>>> will be difficult for certain organisations to adopt and be current with
>>>>> the latest driver.
>>>>>
>>>>>
>>>>>
>>>>> https://docs.datastax.com/en/developer/java-driver/4.9/manual/core/load_balancing/
>>>>>
>>>>> vs
>>>>>
>>>>>
>>>>> https://docs.datastax.com/en/developer/java-driver/3.9/manual/load_balancing/
>>>>>
>>>>>
>>>>> 2 - No more changing consistency on retries
>>>>>
>>>>> Although this has been a controversial feature, we know quite a few
>>>>> users of this feature.
>>>>>
>>>>>
>>>>> Removing this in v4.9 of the driver essentially breaks those
>>>>> applications and it is not possible to implement this through a custom
>>>>> retry policy as the driver RetryDecision is now a String enum (
>>>>> https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/retry/RetryDecision.html)
>>>>> vs v3.9 (
>>>>> https://docs.datastax.com/en/drivers/java/3.9/com/datastax/driver/core/policies/RetryPolicy.RetryDecision.html)
>>>>> where more complex choices could be bubbled up to the driver around the
>>>>> consistency to retry on. We looked into how we could simulate the old retry
>>>>> behaviour using the latest driver and it would require non-trivial code
>>>>> updates within the client application - particularly if the asynchronous
>>>>> features are leveraged.
>>>>>
>>>>>
>>>>> Also, the default settings for the driver has remote connections set
>>>>> to 1 -
>>>>> https://docs.datastax.com/en/developer/java-driver/4.9/manual/core/pooling/#configuration
>>>>> We are not quite certain the reason or think of a scenario when this is
>>>>> utilised as local connections are only possible?
>>>>>
>>>>>
>>>>>
>>>>> https://docs.datastax.com/en/developer/java-driver/4.9/faq/#where-is-downgrading-consistency-retry-policy
>>>>>
>>>>>
>>>>> We look after many DataStax / Cassandra clusters and these changes in
>>>>> the latest driver changes have left us scratching our heads.
>>>>>
>>>>>
>>>>> We would be interested in hearing your thoughts on these observations.
>>>>>
>>>>>
>>>>> Thanks,
>>>>>
>>>>>
>>>>> Johnny
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> Johnny Miller
>>>>>
>>>>> Co-Founder & CTO
>>>>>
>>>>> https://digitalis.io
>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__digitalis.io_&d=DwMFaQ&c=adz96Xi0w1RHqtPMowiL2g&r=CRDTby7ZKMKOJuR7qj8hGNHMqiMtY_3sbc8FBSZtTdo&m=SbjlNK32WNTSbKmszA9g1RNfF-8qJMzwFQIeKJlvAK8&s=wyOKNUxu-__ODwiGCMpfLzJEvGKVCDs5VJ5YqNu8y1Q&e=> |
>>>>> Work: +44 20805023721| Mobile: +352 621159920
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> The information contained in this electronic message and any
>>>>> attachments to this message are intended for the exclusive use of the
>>>>> addressee(s) and may contain proprietary, confidential or privileged
>>>>> information. If you are not the intended recipient, you should not
>>>>> disseminate, distribute or copy this e-mail. Please notify the sender
>>>>> immediately and destroy all copies of this message and any attachments.
>>>>> WARNING: Computer viruses can be transmitted via email. The recipient
>>>>> should check this email and any attachments for the presence of viruses.
>>>>> The company accepts no liability for any damage caused by any virus
>>>>> transmitted by this email. www.digitalis.io
>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.digitalis.io&d=DwMFaQ&c=adz96Xi0w1RHqtPMowiL2g&r=CRDTby7ZKMKOJuR7qj8hGNHMqiMtY_3sbc8FBSZtTdo&m=SbjlNK32WNTSbKmszA9g1RNfF-8qJMzwFQIeKJlvAK8&s=KQnoR-TFsGz9Ln6x1k1pIo7JO1fbpjwZ03JwN19ONtQ&e=>
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>>
>>> The information contained in this electronic message and any attachments
>>> to this message are intended for the exclusive use of the addressee(s) and
>>> may contain proprietary, confidential or privileged information. If you are
>>> not the intended recipient, you should not disseminate, distribute or copy
>>> this e-mail. Please notify the sender immediately and destroy all copies of
>>> this message and any attachments. WARNING: Computer viruses can be
>>> transmitted via email. The recipient should check this email and any
>>> attachments for the presence of viruses. The company accepts no liability
>>> for any damage caused by any virus transmitted by this email.
>>> www.digitalis.io
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.digitalis.io&d=DwMFaQ&c=adz96Xi0w1RHqtPMowiL2g&r=CRDTby7ZKMKOJuR7qj8hGNHMqiMtY_3sbc8FBSZtTdo&m=SbjlNK32WNTSbKmszA9g1RNfF-8qJMzwFQIeKJlvAK8&s=KQnoR-TFsGz9Ln6x1k1pIo7JO1fbpjwZ03JwN19ONtQ&e=>
>>>
>>
--
Cedrick Lunven
e. cedrick.lunven@datastax.com
w. www.datastax.com
Re: What does the community think of the DataStax 4.x Java driver changes?
Posted by Angelo Polo <la...@gmail.com>.
Execution profiles are a brilliant new feature of the 4.x drivers. But
configuring them was a challenge. Programmatic configuration offers few
advantages over the file-based one because options can still only be
specified by name for later injection. When runtime settings are to be
applied, one would like to be able to provide an instantiated object for a
given property (e.g. load balancer). Instead, one must confect a class with
bytecode manipulation, or some other means, and/or push information through
the DriverContext later and have to deal with timing and mutability issues.
It's easy to find oneself deep in the "internal" tree, overriding
components far removed from the original target of the customization.
Best,
Angelo
On Fri, Oct 30, 2020 at 4:18 AM Jonathan Koppenhofer <jo...@koppedomain.com>
wrote:
> We actually feel the same where we have edge cases where downgrading CL
> was useful. We did end up writing this in application logic as our code was
> pretty well abstracted and centralized to do so.
>
> I will also agree the driver seems overly prescriptive now with limited
> ability to override. Good for protecting users from themselves, but bad for
> edge cases.
>
> Generally, we work with alot of application teams, and most of them are
> putting off the inevitable of the changes required to implement 4.x. In a
> reasonably well written moderately complex app, it took one of our best
> developers 2 weeks make the changes, and we were finding bugs for a couple
> releases after that. Not positive experience.
>
> We now are done with the migration, and things are working reasonably well.
>
>
> On Thu, Oct 29, 2020, 9:06 AM Johnny Miller <jo...@digitalis.io> wrote:
>
>> Joshua - thanks for the update, I have found the ASF slack channel
>> (#cassandra-cep-drivers-donation) and google doc (
>> https://docs.google.com/document/d/1e0SsZxjeTabzrMv99pCz9zIkkgWjUd4KL5Yp0GFzNnY/edit#).
>> Will be watching it closely.
>>
>> In terms of the functional changes brought into the driver with 4.x the
>> downgrading CL has always been a controversial feature, but the failover to
>> remote DC being removed - I am curious to understand why?
>>
>> Thanks,
>>
>> Johnny
>>
>> On Thu, 29 Oct 2020 at 13:53, Joshua McKenzie <jm...@apache.org>
>> wrote:
>>
>>> That's an immense amount of incredibly useful feedback Johnny. Thanks
>>> for taking the time and energy to write all this up.
>>>
>>> I work with some of the engineers who authored these changes in the
>>> driver and have brought this thread to their attention. The authors have
>>> offered the driver as a CEP donation to the C* project so we will have one
>>> in tree which should give a clear path to fixing some of these API issues
>>> as well as the loss of functionality on a major.
>>>
>>>
>>> On Thu, Oct 29, 2020 at 8:37 AM, Johnny Miller <jo...@digitalis.io>
>>> wrote:
>>>
>>>> Hi Everybody,
>>>>
>>>>
>>>> We wanted to reach out to the community around the v4 changes in the
>>>> DataStax Java driver and gauge people's opinions on some of the changes.
>>>> DataStax have done a tremendous job over the years on the Cassandra drivers
>>>> and contributing to this community. However, we are currently struggling to
>>>> adopt the latest driver due to these changes.
>>>>
>>>>
>>>> We have been working on a project to upgrade an application from v3 to
>>>> v4.9 of the driver and have encountered major changes between these
>>>> versions.
>>>>
>>>>
>>>> We have observed the latest version of the driver contains many more
>>>> DataStax Enterprise (DSE) specific code, and this is not surprising as
>>>> DataStax have been generous to build it for the Cassandra community.
>>>>
>>>>
>>>> From our understanding, the DSE specific code must be included even if
>>>> you are unable to use it or require it. For example, in CqlSessionBuilder
>>>> class which is the main entry point into the driver, there are APIs
>>>> relating directly to DataStax Enterprise non-OSS functionality, their cloud
>>>> DBaaS etc.. e.g.
>>>>
>>>>
>>>> - withApplicationName (
>>>> https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/session/SessionBuilder.html#withApplicationName-java.lang.String-
>>>> )
>>>>
>>>> - withApplicationVersion (
>>>> https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/session/SessionBuilder.html#withApplicationVersion-java.lang.String-
>>>> )
>>>>
>>>> - withCloudProxyAddress (
>>>> https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/session/SessionBuilder.html#withCloudProxyAddress-java.net.InetSocketAddress-
>>>> )
>>>>
>>>> - withCloudProxyAddress (
>>>> https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/session/SessionBuilder.html#withCloudSecureConnectBundle-java.io.InputStream-
>>>> )
>>>>
>>>>
>>>> plus more.
>>>>
>>>>
>>>> All of these are sitting under the com.datastax.oss package - not the
>>>> com.datastax.dse package.
>>>>
>>>>
>>>> Additionally the reference.conf for the default driver settings
>>>> contains a large number of DSE specific options:
>>>>
>>>>
>>>> https://github.com/datastax/java-driver/blob/4.9.0/core/src/main/resources/reference.conf
>>>>
>>>>
>>>> We would like to have seen this implemented in a subclass of the
>>>> CqlSessionBuilder eg. DataStaxCqlSessionBuilder and the conf split into two
>>>> separate config files.
>>>>
>>>>
>>>> Additionally, the structure of the library is such that it is bundling
>>>> all of the DSE driver code with the non-DSE driver code eg. graph, search
>>>> etc. We would also like to have seen DataStax to have implemented it as
>>>> separate libs and use a dependency on an OSS only lib in the datastax
>>>> specific lib for the shared functionality.
>>>>
>>>>
>>>> It would be great to be able to only take in the dependencies and code
>>>> needed for Apache Cassandra and not the commercial products around it.
>>>>
>>>>
>>>> However, the above observations are trivial compared to the two core
>>>> features of the driver that seem to have been deprecated and we would like
>>>> your opinion.
>>>>
>>>>
>>>> 1 - No more failovers to remote-DC
>>>>
>>>> Previous versions of the driver allowed the driver to failover to a
>>>> remote DC (if you wanted) in the event of losing access to the local DC.
>>>> This is no longer the case.
>>>>
>>>>
>>>> See:
>>>> https://docs.datastax.com/en/developer/java-driver/4.9/manual/core/load_balancing/#default-policy
>>>>
>>>>
>>>> What this means is either:
>>>>
>>>> a - re-implement this - not trivial
>>>>
>>>> b - initialise multiple instances of the driver in your JVM - not
>>>> recommended by DataStax and we have observed some issues when you do this
>>>>
>>>> c - Address the problem via infra e.g load balancer fronting a local
>>>> service connecting to local Cassandra DC along side a passive local service
>>>> connecting to the remote Cassandra DC
>>>>
>>>> d - stop the entire applications/Cassandra stack in the DC if Cassandra
>>>> becomes unavailable to the applications in this DC for any reason.
>>>>
>>>>
>>>> Not a trivial change to take on, and in our humble opinion undermines
>>>> one of the greatest benefits of Apache Cassandra where your applications
>>>> "just keep working". We have seen many situations where this has been one
>>>> of the core architecture principles for adopting Cassandra and forms part
>>>> of the signed off and agreed operational acceptance testing on platforms.
>>>>
>>>>
>>>> The latest driver will not be able to retain this architecture and it
>>>> will be difficult for certain organisations to adopt and be current with
>>>> the latest driver.
>>>>
>>>>
>>>>
>>>> https://docs.datastax.com/en/developer/java-driver/4.9/manual/core/load_balancing/
>>>>
>>>> vs
>>>>
>>>>
>>>> https://docs.datastax.com/en/developer/java-driver/3.9/manual/load_balancing/
>>>>
>>>>
>>>> 2 - No more changing consistency on retries
>>>>
>>>> Although this has been a controversial feature, we know quite a few
>>>> users of this feature.
>>>>
>>>>
>>>> Removing this in v4.9 of the driver essentially breaks those
>>>> applications and it is not possible to implement this through a custom
>>>> retry policy as the driver RetryDecision is now a String enum (
>>>> https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/retry/RetryDecision.html)
>>>> vs v3.9 (
>>>> https://docs.datastax.com/en/drivers/java/3.9/com/datastax/driver/core/policies/RetryPolicy.RetryDecision.html)
>>>> where more complex choices could be bubbled up to the driver around the
>>>> consistency to retry on. We looked into how we could simulate the old retry
>>>> behaviour using the latest driver and it would require non-trivial code
>>>> updates within the client application - particularly if the asynchronous
>>>> features are leveraged.
>>>>
>>>>
>>>> Also, the default settings for the driver has remote connections set to
>>>> 1 -
>>>> https://docs.datastax.com/en/developer/java-driver/4.9/manual/core/pooling/#configuration
>>>> We are not quite certain the reason or think of a scenario when this is
>>>> utilised as local connections are only possible?
>>>>
>>>>
>>>>
>>>> https://docs.datastax.com/en/developer/java-driver/4.9/faq/#where-is-downgrading-consistency-retry-policy
>>>>
>>>>
>>>> We look after many DataStax / Cassandra clusters and these changes in
>>>> the latest driver changes have left us scratching our heads.
>>>>
>>>>
>>>> We would be interested in hearing your thoughts on these observations.
>>>>
>>>>
>>>> Thanks,
>>>>
>>>>
>>>> Johnny
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Johnny Miller
>>>>
>>>> Co-Founder & CTO
>>>>
>>>> https://digitalis.io <http://digitalis.io/> | Work: +44 20805023721|
>>>> Mobile: +352 621159920
>>>>
>>>>
>>>> --
>>>>
>>>> The information contained in this electronic message and any
>>>> attachments to this message are intended for the exclusive use of the
>>>> addressee(s) and may contain proprietary, confidential or privileged
>>>> information. If you are not the intended recipient, you should not
>>>> disseminate, distribute or copy this e-mail. Please notify the sender
>>>> immediately and destroy all copies of this message and any attachments.
>>>> WARNING: Computer viruses can be transmitted via email. The recipient
>>>> should check this email and any attachments for the presence of viruses.
>>>> The company accepts no liability for any damage caused by any virus
>>>> transmitted by this email. www.digitalis.io
>>>>
>>>
>>>
>>
>>
>> --
>>
>> The information contained in this electronic message and any attachments
>> to this message are intended for the exclusive use of the addressee(s) and
>> may contain proprietary, confidential or privileged information. If you are
>> not the intended recipient, you should not disseminate, distribute or copy
>> this e-mail. Please notify the sender immediately and destroy all copies of
>> this message and any attachments. WARNING: Computer viruses can be
>> transmitted via email. The recipient should check this email and any
>> attachments for the presence of viruses. The company accepts no liability
>> for any damage caused by any virus transmitted by this email.
>> www.digitalis.io
>>
>
Re: What does the community think of the DataStax 4.x Java driver changes?
Posted by Jonathan Koppenhofer <jo...@koppedomain.com>.
We actually feel the same where we have edge cases where downgrading CL was
useful. We did end up writing this in application logic as our code was
pretty well abstracted and centralized to do so.
I will also agree the driver seems overly prescriptive now with limited
ability to override. Good for protecting users from themselves, but bad for
edge cases.
Generally, we work with alot of application teams, and most of them are
putting off the inevitable of the changes required to implement 4.x. In a
reasonably well written moderately complex app, it took one of our best
developers 2 weeks make the changes, and we were finding bugs for a couple
releases after that. Not positive experience.
We now are done with the migration, and things are working reasonably well.
On Thu, Oct 29, 2020, 9:06 AM Johnny Miller <jo...@digitalis.io> wrote:
> Joshua - thanks for the update, I have found the ASF slack channel
> (#cassandra-cep-drivers-donation) and google doc (
> https://docs.google.com/document/d/1e0SsZxjeTabzrMv99pCz9zIkkgWjUd4KL5Yp0GFzNnY/edit#).
> Will be watching it closely.
>
> In terms of the functional changes brought into the driver with 4.x the
> downgrading CL has always been a controversial feature, but the failover to
> remote DC being removed - I am curious to understand why?
>
> Thanks,
>
> Johnny
>
> On Thu, 29 Oct 2020 at 13:53, Joshua McKenzie <jm...@apache.org>
> wrote:
>
>> That's an immense amount of incredibly useful feedback Johnny. Thanks for
>> taking the time and energy to write all this up.
>>
>> I work with some of the engineers who authored these changes in the
>> driver and have brought this thread to their attention. The authors have
>> offered the driver as a CEP donation to the C* project so we will have one
>> in tree which should give a clear path to fixing some of these API issues
>> as well as the loss of functionality on a major.
>>
>>
>> On Thu, Oct 29, 2020 at 8:37 AM, Johnny Miller <jo...@digitalis.io>
>> wrote:
>>
>>> Hi Everybody,
>>>
>>>
>>> We wanted to reach out to the community around the v4 changes in the
>>> DataStax Java driver and gauge people's opinions on some of the changes.
>>> DataStax have done a tremendous job over the years on the Cassandra drivers
>>> and contributing to this community. However, we are currently struggling to
>>> adopt the latest driver due to these changes.
>>>
>>>
>>> We have been working on a project to upgrade an application from v3 to
>>> v4.9 of the driver and have encountered major changes between these
>>> versions.
>>>
>>>
>>> We have observed the latest version of the driver contains many more
>>> DataStax Enterprise (DSE) specific code, and this is not surprising as
>>> DataStax have been generous to build it for the Cassandra community.
>>>
>>>
>>> From our understanding, the DSE specific code must be included even if
>>> you are unable to use it or require it. For example, in CqlSessionBuilder
>>> class which is the main entry point into the driver, there are APIs
>>> relating directly to DataStax Enterprise non-OSS functionality, their cloud
>>> DBaaS etc.. e.g.
>>>
>>>
>>> - withApplicationName (
>>> https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/session/SessionBuilder.html#withApplicationName-java.lang.String-
>>> )
>>>
>>> - withApplicationVersion (
>>> https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/session/SessionBuilder.html#withApplicationVersion-java.lang.String-
>>> )
>>>
>>> - withCloudProxyAddress (
>>> https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/session/SessionBuilder.html#withCloudProxyAddress-java.net.InetSocketAddress-
>>> )
>>>
>>> - withCloudProxyAddress (
>>> https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/session/SessionBuilder.html#withCloudSecureConnectBundle-java.io.InputStream-
>>> )
>>>
>>>
>>> plus more.
>>>
>>>
>>> All of these are sitting under the com.datastax.oss package - not the
>>> com.datastax.dse package.
>>>
>>>
>>> Additionally the reference.conf for the default driver settings contains
>>> a large number of DSE specific options:
>>>
>>>
>>> https://github.com/datastax/java-driver/blob/4.9.0/core/src/main/resources/reference.conf
>>>
>>>
>>> We would like to have seen this implemented in a subclass of the
>>> CqlSessionBuilder eg. DataStaxCqlSessionBuilder and the conf split into two
>>> separate config files.
>>>
>>>
>>> Additionally, the structure of the library is such that it is bundling
>>> all of the DSE driver code with the non-DSE driver code eg. graph, search
>>> etc. We would also like to have seen DataStax to have implemented it as
>>> separate libs and use a dependency on an OSS only lib in the datastax
>>> specific lib for the shared functionality.
>>>
>>>
>>> It would be great to be able to only take in the dependencies and code
>>> needed for Apache Cassandra and not the commercial products around it.
>>>
>>>
>>> However, the above observations are trivial compared to the two core
>>> features of the driver that seem to have been deprecated and we would like
>>> your opinion.
>>>
>>>
>>> 1 - No more failovers to remote-DC
>>>
>>> Previous versions of the driver allowed the driver to failover to a
>>> remote DC (if you wanted) in the event of losing access to the local DC.
>>> This is no longer the case.
>>>
>>>
>>> See:
>>> https://docs.datastax.com/en/developer/java-driver/4.9/manual/core/load_balancing/#default-policy
>>>
>>>
>>> What this means is either:
>>>
>>> a - re-implement this - not trivial
>>>
>>> b - initialise multiple instances of the driver in your JVM - not
>>> recommended by DataStax and we have observed some issues when you do this
>>>
>>> c - Address the problem via infra e.g load balancer fronting a local
>>> service connecting to local Cassandra DC along side a passive local service
>>> connecting to the remote Cassandra DC
>>>
>>> d - stop the entire applications/Cassandra stack in the DC if Cassandra
>>> becomes unavailable to the applications in this DC for any reason.
>>>
>>>
>>> Not a trivial change to take on, and in our humble opinion undermines
>>> one of the greatest benefits of Apache Cassandra where your applications
>>> "just keep working". We have seen many situations where this has been one
>>> of the core architecture principles for adopting Cassandra and forms part
>>> of the signed off and agreed operational acceptance testing on platforms.
>>>
>>>
>>> The latest driver will not be able to retain this architecture and it
>>> will be difficult for certain organisations to adopt and be current with
>>> the latest driver.
>>>
>>>
>>>
>>> https://docs.datastax.com/en/developer/java-driver/4.9/manual/core/load_balancing/
>>>
>>> vs
>>>
>>>
>>> https://docs.datastax.com/en/developer/java-driver/3.9/manual/load_balancing/
>>>
>>>
>>> 2 - No more changing consistency on retries
>>>
>>> Although this has been a controversial feature, we know quite a few
>>> users of this feature.
>>>
>>>
>>> Removing this in v4.9 of the driver essentially breaks those
>>> applications and it is not possible to implement this through a custom
>>> retry policy as the driver RetryDecision is now a String enum (
>>> https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/retry/RetryDecision.html)
>>> vs v3.9 (
>>> https://docs.datastax.com/en/drivers/java/3.9/com/datastax/driver/core/policies/RetryPolicy.RetryDecision.html)
>>> where more complex choices could be bubbled up to the driver around the
>>> consistency to retry on. We looked into how we could simulate the old retry
>>> behaviour using the latest driver and it would require non-trivial code
>>> updates within the client application - particularly if the asynchronous
>>> features are leveraged.
>>>
>>>
>>> Also, the default settings for the driver has remote connections set to
>>> 1 -
>>> https://docs.datastax.com/en/developer/java-driver/4.9/manual/core/pooling/#configuration
>>> We are not quite certain the reason or think of a scenario when this is
>>> utilised as local connections are only possible?
>>>
>>>
>>>
>>> https://docs.datastax.com/en/developer/java-driver/4.9/faq/#where-is-downgrading-consistency-retry-policy
>>>
>>>
>>> We look after many DataStax / Cassandra clusters and these changes in
>>> the latest driver changes have left us scratching our heads.
>>>
>>>
>>> We would be interested in hearing your thoughts on these observations.
>>>
>>>
>>> Thanks,
>>>
>>>
>>> Johnny
>>>
>>>
>>>
>>>
>>> --
>>>
>>> Johnny Miller
>>>
>>> Co-Founder & CTO
>>>
>>> https://digitalis.io <http://digitalis.io/> | Work: +44 20805023721|
>>> Mobile: +352 621159920
>>>
>>>
>>> --
>>>
>>> The information contained in this electronic message and any attachments
>>> to this message are intended for the exclusive use of the addressee(s) and
>>> may contain proprietary, confidential or privileged information. If you are
>>> not the intended recipient, you should not disseminate, distribute or copy
>>> this e-mail. Please notify the sender immediately and destroy all copies of
>>> this message and any attachments. WARNING: Computer viruses can be
>>> transmitted via email. The recipient should check this email and any
>>> attachments for the presence of viruses. The company accepts no liability
>>> for any damage caused by any virus transmitted by this email.
>>> www.digitalis.io
>>>
>>
>>
>
>
> --
>
> The information contained in this electronic message and any attachments
> to this message are intended for the exclusive use of the addressee(s) and
> may contain proprietary, confidential or privileged information. If you are
> not the intended recipient, you should not disseminate, distribute or copy
> this e-mail. Please notify the sender immediately and destroy all copies of
> this message and any attachments. WARNING: Computer viruses can be
> transmitted via email. The recipient should check this email and any
> attachments for the presence of viruses. The company accepts no liability
> for any damage caused by any virus transmitted by this email.
> www.digitalis.io
>
Re: What does the community think of the DataStax 4.x Java driver changes?
Posted by DuyHai Doan <do...@gmail.com>.
Just my 2 cents
Because of the tremendous breaking changes in terms of API as well as
public facing classes (QueryBuilder for ex) I have stopped the development
of the Achilles framework.
Migrating to the 4.x version would require almost the complete rewrite of
the framework, an effort which I cannot afford to dedicate to (the
framework is 7 years old now). I also advise many customers of mine to
adopt a wait & see strategy with regards to the new drivers because of the
amount of application rewrite, due to the aforementioned public-facing
classes changes
Regards
Duy Hai DOAN
On Thu, Oct 29, 2020 at 2:06 PM Johnny Miller <jo...@digitalis.io> wrote:
> Joshua - thanks for the update, I have found the ASF slack channel
> (#cassandra-cep-drivers-donation) and google doc (
> https://docs.google.com/document/d/1e0SsZxjeTabzrMv99pCz9zIkkgWjUd4KL5Yp0GFzNnY/edit#).
> Will be watching it closely.
>
> In terms of the functional changes brought into the driver with 4.x the
> downgrading CL has always been a controversial feature, but the failover to
> remote DC being removed - I am curious to understand why?
>
> Thanks,
>
> Johnny
>
> On Thu, 29 Oct 2020 at 13:53, Joshua McKenzie <jm...@apache.org>
> wrote:
>
>> That's an immense amount of incredibly useful feedback Johnny. Thanks for
>> taking the time and energy to write all this up.
>>
>> I work with some of the engineers who authored these changes in the
>> driver and have brought this thread to their attention. The authors have
>> offered the driver as a CEP donation to the C* project so we will have one
>> in tree which should give a clear path to fixing some of these API issues
>> as well as the loss of functionality on a major.
>>
>>
>> On Thu, Oct 29, 2020 at 8:37 AM, Johnny Miller <jo...@digitalis.io>
>> wrote:
>>
>>> Hi Everybody,
>>>
>>>
>>> We wanted to reach out to the community around the v4 changes in the
>>> DataStax Java driver and gauge people's opinions on some of the changes.
>>> DataStax have done a tremendous job over the years on the Cassandra drivers
>>> and contributing to this community. However, we are currently struggling to
>>> adopt the latest driver due to these changes.
>>>
>>>
>>> We have been working on a project to upgrade an application from v3 to
>>> v4.9 of the driver and have encountered major changes between these
>>> versions.
>>>
>>>
>>> We have observed the latest version of the driver contains many more
>>> DataStax Enterprise (DSE) specific code, and this is not surprising as
>>> DataStax have been generous to build it for the Cassandra community.
>>>
>>>
>>> From our understanding, the DSE specific code must be included even if
>>> you are unable to use it or require it. For example, in CqlSessionBuilder
>>> class which is the main entry point into the driver, there are APIs
>>> relating directly to DataStax Enterprise non-OSS functionality, their cloud
>>> DBaaS etc.. e.g.
>>>
>>>
>>> - withApplicationName (
>>> https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/session/SessionBuilder.html#withApplicationName-java.lang.String-
>>> )
>>>
>>> - withApplicationVersion (
>>> https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/session/SessionBuilder.html#withApplicationVersion-java.lang.String-
>>> )
>>>
>>> - withCloudProxyAddress (
>>> https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/session/SessionBuilder.html#withCloudProxyAddress-java.net.InetSocketAddress-
>>> )
>>>
>>> - withCloudProxyAddress (
>>> https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/session/SessionBuilder.html#withCloudSecureConnectBundle-java.io.InputStream-
>>> )
>>>
>>>
>>> plus more.
>>>
>>>
>>> All of these are sitting under the com.datastax.oss package - not the
>>> com.datastax.dse package.
>>>
>>>
>>> Additionally the reference.conf for the default driver settings contains
>>> a large number of DSE specific options:
>>>
>>>
>>> https://github.com/datastax/java-driver/blob/4.9.0/core/src/main/resources/reference.conf
>>>
>>>
>>> We would like to have seen this implemented in a subclass of the
>>> CqlSessionBuilder eg. DataStaxCqlSessionBuilder and the conf split into two
>>> separate config files.
>>>
>>>
>>> Additionally, the structure of the library is such that it is bundling
>>> all of the DSE driver code with the non-DSE driver code eg. graph, search
>>> etc. We would also like to have seen DataStax to have implemented it as
>>> separate libs and use a dependency on an OSS only lib in the datastax
>>> specific lib for the shared functionality.
>>>
>>>
>>> It would be great to be able to only take in the dependencies and code
>>> needed for Apache Cassandra and not the commercial products around it.
>>>
>>>
>>> However, the above observations are trivial compared to the two core
>>> features of the driver that seem to have been deprecated and we would like
>>> your opinion.
>>>
>>>
>>> 1 - No more failovers to remote-DC
>>>
>>> Previous versions of the driver allowed the driver to failover to a
>>> remote DC (if you wanted) in the event of losing access to the local DC.
>>> This is no longer the case.
>>>
>>>
>>> See:
>>> https://docs.datastax.com/en/developer/java-driver/4.9/manual/core/load_balancing/#default-policy
>>>
>>>
>>> What this means is either:
>>>
>>> a - re-implement this - not trivial
>>>
>>> b - initialise multiple instances of the driver in your JVM - not
>>> recommended by DataStax and we have observed some issues when you do this
>>>
>>> c - Address the problem via infra e.g load balancer fronting a local
>>> service connecting to local Cassandra DC along side a passive local service
>>> connecting to the remote Cassandra DC
>>>
>>> d - stop the entire applications/Cassandra stack in the DC if Cassandra
>>> becomes unavailable to the applications in this DC for any reason.
>>>
>>>
>>> Not a trivial change to take on, and in our humble opinion undermines
>>> one of the greatest benefits of Apache Cassandra where your applications
>>> "just keep working". We have seen many situations where this has been one
>>> of the core architecture principles for adopting Cassandra and forms part
>>> of the signed off and agreed operational acceptance testing on platforms.
>>>
>>>
>>> The latest driver will not be able to retain this architecture and it
>>> will be difficult for certain organisations to adopt and be current with
>>> the latest driver.
>>>
>>>
>>>
>>> https://docs.datastax.com/en/developer/java-driver/4.9/manual/core/load_balancing/
>>>
>>> vs
>>>
>>>
>>> https://docs.datastax.com/en/developer/java-driver/3.9/manual/load_balancing/
>>>
>>>
>>> 2 - No more changing consistency on retries
>>>
>>> Although this has been a controversial feature, we know quite a few
>>> users of this feature.
>>>
>>>
>>> Removing this in v4.9 of the driver essentially breaks those
>>> applications and it is not possible to implement this through a custom
>>> retry policy as the driver RetryDecision is now a String enum (
>>> https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/retry/RetryDecision.html)
>>> vs v3.9 (
>>> https://docs.datastax.com/en/drivers/java/3.9/com/datastax/driver/core/policies/RetryPolicy.RetryDecision.html)
>>> where more complex choices could be bubbled up to the driver around the
>>> consistency to retry on. We looked into how we could simulate the old retry
>>> behaviour using the latest driver and it would require non-trivial code
>>> updates within the client application - particularly if the asynchronous
>>> features are leveraged.
>>>
>>>
>>> Also, the default settings for the driver has remote connections set to
>>> 1 -
>>> https://docs.datastax.com/en/developer/java-driver/4.9/manual/core/pooling/#configuration
>>> We are not quite certain the reason or think of a scenario when this is
>>> utilised as local connections are only possible?
>>>
>>>
>>>
>>> https://docs.datastax.com/en/developer/java-driver/4.9/faq/#where-is-downgrading-consistency-retry-policy
>>>
>>>
>>> We look after many DataStax / Cassandra clusters and these changes in
>>> the latest driver changes have left us scratching our heads.
>>>
>>>
>>> We would be interested in hearing your thoughts on these observations.
>>>
>>>
>>> Thanks,
>>>
>>>
>>> Johnny
>>>
>>>
>>>
>>>
>>> --
>>>
>>> Johnny Miller
>>>
>>> Co-Founder & CTO
>>>
>>> https://digitalis.io <http://digitalis.io/> | Work: +44 20805023721|
>>> Mobile: +352 621159920
>>>
>>>
>>> --
>>>
>>> The information contained in this electronic message and any attachments
>>> to this message are intended for the exclusive use of the addressee(s) and
>>> may contain proprietary, confidential or privileged information. If you are
>>> not the intended recipient, you should not disseminate, distribute or copy
>>> this e-mail. Please notify the sender immediately and destroy all copies of
>>> this message and any attachments. WARNING: Computer viruses can be
>>> transmitted via email. The recipient should check this email and any
>>> attachments for the presence of viruses. The company accepts no liability
>>> for any damage caused by any virus transmitted by this email.
>>> www.digitalis.io
>>>
>>
>>
>
>
> --
>
> The information contained in this electronic message and any attachments
> to this message are intended for the exclusive use of the addressee(s) and
> may contain proprietary, confidential or privileged information. If you are
> not the intended recipient, you should not disseminate, distribute or copy
> this e-mail. Please notify the sender immediately and destroy all copies of
> this message and any attachments. WARNING: Computer viruses can be
> transmitted via email. The recipient should check this email and any
> attachments for the presence of viruses. The company accepts no liability
> for any damage caused by any virus transmitted by this email.
> www.digitalis.io
>
Re: What does the community think of the DataStax 4.x Java driver changes?
Posted by Johnny Miller <jo...@digitalis.io>.
Joshua - thanks for the update, I have found the ASF slack channel
(#cassandra-cep-drivers-donation) and google doc (
https://docs.google.com/document/d/1e0SsZxjeTabzrMv99pCz9zIkkgWjUd4KL5Yp0GFzNnY/edit#).
Will be watching it closely.
In terms of the functional changes brought into the driver with 4.x the
downgrading CL has always been a controversial feature, but the failover to
remote DC being removed - I am curious to understand why?
Thanks,
Johnny
On Thu, 29 Oct 2020 at 13:53, Joshua McKenzie <jm...@apache.org> wrote:
> That's an immense amount of incredibly useful feedback Johnny. Thanks for
> taking the time and energy to write all this up.
>
> I work with some of the engineers who authored these changes in the driver
> and have brought this thread to their attention. The authors have offered
> the driver as a CEP donation to the C* project so we will have one in tree
> which should give a clear path to fixing some of these API issues as well
> as the loss of functionality on a major.
>
>
> On Thu, Oct 29, 2020 at 8:37 AM, Johnny Miller <jo...@digitalis.io>
> wrote:
>
>> Hi Everybody,
>>
>>
>> We wanted to reach out to the community around the v4 changes in the
>> DataStax Java driver and gauge people's opinions on some of the changes.
>> DataStax have done a tremendous job over the years on the Cassandra drivers
>> and contributing to this community. However, we are currently struggling to
>> adopt the latest driver due to these changes.
>>
>>
>> We have been working on a project to upgrade an application from v3 to
>> v4.9 of the driver and have encountered major changes between these
>> versions.
>>
>>
>> We have observed the latest version of the driver contains many more
>> DataStax Enterprise (DSE) specific code, and this is not surprising as
>> DataStax have been generous to build it for the Cassandra community.
>>
>>
>> From our understanding, the DSE specific code must be included even if
>> you are unable to use it or require it. For example, in CqlSessionBuilder
>> class which is the main entry point into the driver, there are APIs
>> relating directly to DataStax Enterprise non-OSS functionality, their cloud
>> DBaaS etc.. e.g.
>>
>>
>> - withApplicationName (
>> https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/session/SessionBuilder.html#withApplicationName-java.lang.String-
>> )
>>
>> - withApplicationVersion (
>> https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/session/SessionBuilder.html#withApplicationVersion-java.lang.String-
>> )
>>
>> - withCloudProxyAddress (
>> https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/session/SessionBuilder.html#withCloudProxyAddress-java.net.InetSocketAddress-
>> )
>>
>> - withCloudProxyAddress (
>> https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/session/SessionBuilder.html#withCloudSecureConnectBundle-java.io.InputStream-
>> )
>>
>>
>> plus more.
>>
>>
>> All of these are sitting under the com.datastax.oss package - not the
>> com.datastax.dse package.
>>
>>
>> Additionally the reference.conf for the default driver settings contains
>> a large number of DSE specific options:
>>
>>
>> https://github.com/datastax/java-driver/blob/4.9.0/core/src/main/resources/reference.conf
>>
>>
>> We would like to have seen this implemented in a subclass of the
>> CqlSessionBuilder eg. DataStaxCqlSessionBuilder and the conf split into two
>> separate config files.
>>
>>
>> Additionally, the structure of the library is such that it is bundling
>> all of the DSE driver code with the non-DSE driver code eg. graph, search
>> etc. We would also like to have seen DataStax to have implemented it as
>> separate libs and use a dependency on an OSS only lib in the datastax
>> specific lib for the shared functionality.
>>
>>
>> It would be great to be able to only take in the dependencies and code
>> needed for Apache Cassandra and not the commercial products around it.
>>
>>
>> However, the above observations are trivial compared to the two core
>> features of the driver that seem to have been deprecated and we would like
>> your opinion.
>>
>>
>> 1 - No more failovers to remote-DC
>>
>> Previous versions of the driver allowed the driver to failover to a
>> remote DC (if you wanted) in the event of losing access to the local DC.
>> This is no longer the case.
>>
>>
>> See:
>> https://docs.datastax.com/en/developer/java-driver/4.9/manual/core/load_balancing/#default-policy
>>
>>
>> What this means is either:
>>
>> a - re-implement this - not trivial
>>
>> b - initialise multiple instances of the driver in your JVM - not
>> recommended by DataStax and we have observed some issues when you do this
>>
>> c - Address the problem via infra e.g load balancer fronting a local
>> service connecting to local Cassandra DC along side a passive local service
>> connecting to the remote Cassandra DC
>>
>> d - stop the entire applications/Cassandra stack in the DC if Cassandra
>> becomes unavailable to the applications in this DC for any reason.
>>
>>
>> Not a trivial change to take on, and in our humble opinion undermines one
>> of the greatest benefits of Apache Cassandra where your applications "just
>> keep working". We have seen many situations where this has been one of the
>> core architecture principles for adopting Cassandra and forms part of the
>> signed off and agreed operational acceptance testing on platforms.
>>
>>
>> The latest driver will not be able to retain this architecture and it
>> will be difficult for certain organisations to adopt and be current with
>> the latest driver.
>>
>>
>>
>> https://docs.datastax.com/en/developer/java-driver/4.9/manual/core/load_balancing/
>>
>> vs
>>
>>
>> https://docs.datastax.com/en/developer/java-driver/3.9/manual/load_balancing/
>>
>>
>> 2 - No more changing consistency on retries
>>
>> Although this has been a controversial feature, we know quite a few users
>> of this feature.
>>
>>
>> Removing this in v4.9 of the driver essentially breaks those applications
>> and it is not possible to implement this through a custom retry policy as
>> the driver RetryDecision is now a String enum (
>> https://docs.datastax.com/en/drivers/java/4.9/com/datastax/oss/driver/api/core/retry/RetryDecision.html)
>> vs v3.9 (
>> https://docs.datastax.com/en/drivers/java/3.9/com/datastax/driver/core/policies/RetryPolicy.RetryDecision.html)
>> where more complex choices could be bubbled up to the driver around the
>> consistency to retry on. We looked into how we could simulate the old retry
>> behaviour using the latest driver and it would require non-trivial code
>> updates within the client application - particularly if the asynchronous
>> features are leveraged.
>>
>>
>> Also, the default settings for the driver has remote connections set to 1
>> -
>> https://docs.datastax.com/en/developer/java-driver/4.9/manual/core/pooling/#configuration
>> We are not quite certain the reason or think of a scenario when this is
>> utilised as local connections are only possible?
>>
>>
>>
>> https://docs.datastax.com/en/developer/java-driver/4.9/faq/#where-is-downgrading-consistency-retry-policy
>>
>>
>> We look after many DataStax / Cassandra clusters and these changes in the
>> latest driver changes have left us scratching our heads.
>>
>>
>> We would be interested in hearing your thoughts on these observations.
>>
>>
>> Thanks,
>>
>>
>> Johnny
>>
>>
>>
>>
>> --
>>
>> Johnny Miller
>>
>> Co-Founder & CTO
>>
>> https://digitalis.io <http://digitalis.io/> | Work: +44 20805023721|
>> Mobile: +352 621159920
>>
>>
>> --
>>
>> The information contained in this electronic message and any attachments
>> to this message are intended for the exclusive use of the addressee(s) and
>> may contain proprietary, confidential or privileged information. If you are
>> not the intended recipient, you should not disseminate, distribute or copy
>> this e-mail. Please notify the sender immediately and destroy all copies of
>> this message and any attachments. WARNING: Computer viruses can be
>> transmitted via email. The recipient should check this email and any
>> attachments for the presence of viruses. The company accepts no liability
>> for any damage caused by any virus transmitted by this email.
>> www.digitalis.io
>>
>
>
--
--
The information contained in this electronic message and any
attachments to this message are intended for the exclusive use of the
addressee(s) and may contain proprietary, confidential or privileged
information. If you are not the intended recipient, you should not
disseminate, distribute or copy this e-mail. Please notify the sender
immediately and destroy all copies of this message and any attachments.
WARNING: Computer viruses can be transmitted via email. The recipient
should check this email and any attachments for the presence of viruses.
The company accepts no liability for any damage caused by any virus
transmitted by this email. www.digitalis.io <http://www.digitalis.io>
Re: What does the community think of the DataStax 4.x Java driver changes?
Posted by Joshua McKenzie <jm...@apache.org>.
That's an immense amount of incredibly useful feedback Johnny. Thanks for
taking the time and energy to write all this up.
I work with some of the engineers who authored these changes in the driver
and have brought this thread to their attention. The authors have offered
the driver as a CEP donation to the C* project so we will have one in tree
which should give a clear path to fixing some of these API issues as well
as the loss of functionality on a major.
On Thu, Oct 29, 2020 at 8:37 AM, Johnny Miller <jo...@digitalis.io> wrote:
> Hi Everybody,
>
>
> We wanted to reach out to the community around the v4 changes in the
> DataStax Java driver and gauge people's opinions on some of the changes.
> DataStax have done a tremendous job over the years on the Cassandra drivers
> and contributing to this community. However, we are currently struggling to
> adopt the latest driver due to these changes.
>
>
> We have been working on a project to upgrade an application from v3 to
> v4.9 of the driver and have encountered major changes between these
> versions.
>
>
> We have observed the latest version of the driver contains many more
> DataStax Enterprise (DSE) specific code, and this is not surprising as
> DataStax have been generous to build it for the Cassandra community.
>
>
> From our understanding, the DSE specific code must be included even if you
> are unable to use it or require it. For example, in CqlSessionBuilder class
> which is the main entry point into the driver, there are APIs relating
> directly to DataStax Enterprise non-OSS functionality, their cloud DBaaS
> etc.. e.g.
>
>
> - withApplicationName (https://docs.datastax.com/en/drivers/java/4.9/com/
> datastax/oss/driver/api/core/session/SessionBuilder.
> html#withApplicationName-java.lang.String-)
>
> - withApplicationVersion (https://docs.datastax.com/en/drivers/java/4.9/
> com/datastax/oss/driver/api/core/session/SessionBuilder.
> html#withApplicationVersion-java.lang.String-)
>
> - withCloudProxyAddress (https://docs.datastax.com/en/drivers/java/4.9/
> com/datastax/oss/driver/api/core/session/SessionBuilder.
> html#withCloudProxyAddress-java.net.InetSocketAddress-)
>
> - withCloudProxyAddress (https://docs.datastax.com/en/drivers/java/4.9/
> com/datastax/oss/driver/api/core/session/SessionBuilder.
> html#withCloudSecureConnectBundle-java.io.InputStream-)
>
>
> plus more.
>
>
> All of these are sitting under the com.datastax.oss package - not the
> com.datastax.dse package.
>
>
> Additionally the reference.conf for the default driver settings contains a
> large number of DSE specific options:
>
> https://github.com/datastax/java-driver/blob/4.9.0/core/src/main/
> resources/reference.conf
>
>
> We would like to have seen this implemented in a subclass of the
> CqlSessionBuilder eg. DataStaxCqlSessionBuilder and the conf split into two
> separate config files.
>
>
> Additionally, the structure of the library is such that it is bundling all
> of the DSE driver code with the non-DSE driver code eg. graph, search etc.
> We would also like to have seen DataStax to have implemented it as separate
> libs and use a dependency on an OSS only lib in the datastax specific lib
> for the shared functionality.
>
>
> It would be great to be able to only take in the dependencies and code
> needed for Apache Cassandra and not the commercial products around it.
>
>
> However, the above observations are trivial compared to the two core
> features of the driver that seem to have been deprecated and we would like
> your opinion.
>
>
> 1 - No more failovers to remote-DC
>
> Previous versions of the driver allowed the driver to failover to a remote
> DC (if you wanted) in the event of losing access to the local DC. This is
> no longer the case.
>
>
> See: https://docs.datastax.com/en/developer/java-driver/4.9/manual/core/
> load_balancing/#default-policy
>
>
> What this means is either:
>
> a - re-implement this - not trivial
>
> b - initialise multiple instances of the driver in your JVM - not
> recommended by DataStax and we have observed some issues when you do this
>
> c - Address the problem via infra e.g load balancer fronting a local
> service connecting to local Cassandra DC along side a passive local service
> connecting to the remote Cassandra DC
>
> d - stop the entire applications/Cassandra stack in the DC if Cassandra
> becomes unavailable to the applications in this DC for any reason.
>
>
> Not a trivial change to take on, and in our humble opinion undermines one
> of the greatest benefits of Apache Cassandra where your applications "just
> keep working". We have seen many situations where this has been one of the
> core architecture principles for adopting Cassandra and forms part of the
> signed off and agreed operational acceptance testing on platforms.
>
>
> The latest driver will not be able to retain this architecture and it will
> be difficult for certain organisations to adopt and be current with the
> latest driver.
>
>
> https://docs.datastax.com/en/developer/java-driver/4.9/manual/core/
> load_balancing/
>
> vs
>
> https://docs.datastax.com/en/developer/java-driver/3.9/manual/
> load_balancing/
>
>
> 2 - No more changing consistency on retries
>
> Although this has been a controversial feature, we know quite a few users
> of this feature.
>
>
> Removing this in v4.9 of the driver essentially breaks those applications
> and it is not possible to implement this through a custom retry policy as
> the driver RetryDecision is now a String enum (https://docs.datastax.com/
> en/drivers/java/4.9/com/datastax/oss/driver/api/core/retry/RetryDecision.
> html) vs v3.9 (https://docs.datastax.com/en/drivers/java/3.9/com/datastax/
> driver/core/policies/RetryPolicy.RetryDecision.html) where more complex
> choices could be bubbled up to the driver around the consistency to retry
> on. We looked into how we could simulate the old retry behaviour using the
> latest driver and it would require non-trivial code updates within the
> client application - particularly if the asynchronous features are
> leveraged.
>
>
> Also, the default settings for the driver has remote connections set to 1
> - https://docs.datastax.com/en/developer/java-driver/4.9/manual/core/
> pooling/#configuration We are not quite certain the reason or think of a
> scenario when this is utilised as local connections are only possible?
>
>
> https://docs.datastax.com/en/developer/java-driver/4.9/faq/
> #where-is-downgrading-consistency-retry-policy
>
>
> We look after many DataStax / Cassandra clusters and these changes in the
> latest driver changes have left us scratching our heads.
>
>
> We would be interested in hearing your thoughts on these observations.
>
>
> Thanks,
>
>
> Johnny
>
>
>
>
> --
>
> Johnny Miller
>
> Co-Founder & CTO
>
> https://digitalis.io <http://digitalis.io/> | Work: +44 20805023721|
> Mobile: +352 621159920
>
>
> --
>
> The information contained in this electronic message and any attachments
> to this message are intended for the exclusive use of the addressee(s) and
> may contain proprietary, confidential or privileged information. If you are
> not the intended recipient, you should not disseminate, distribute or copy
> this e-mail. Please notify the sender immediately and destroy all copies of
> this message and any attachments. WARNING: Computer viruses can be
> transmitted via email. The recipient should check this email and any
> attachments for the presence of viruses. The company accepts no liability
> for any damage caused by any virus transmitted by this email. www.
> digitalis.io
>