You are viewing a plain text version of this content. The canonical link for it is here.
Posted to pr@cassandra.apache.org by "jahstreet (via GitHub)" <gi...@apache.org> on 2025/02/10 11:22:31 UTC
[PR] Add SubnetAddressTranslator [cassandra-java-driver]
jahstreet opened a new pull request, #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013
When running Cassandra in a private network/subnet and accessing it via some kind of proxy, we have an option to use FixedHostNameAddressTranslator. But when we want to set it up in a HA way and have more control over latencies in multi-datacenter deployments, that is not enough.
This PR proposes a SubnetAddressTranslator, which translates Cassandra node IP addresses based on the match to the configured subnet IP range (CIDR notation). The assumption is that each Cassandra datacenter nodes belong to different subnets not having intersecting IP ranges, which is the usual configuration for Kubernetes and K8ssandra, for example.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#discussion_r1949652849
##########
core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/SubnetAddressTranslator.java:
##########
@@ -0,0 +1,210 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.datastax.oss.driver.internal.core.addresstranslation;
+
+import com.datastax.oss.driver.api.core.addresstranslation.AddressTranslator;
+import com.datastax.oss.driver.api.core.config.DriverOption;
+import com.datastax.oss.driver.api.core.context.DriverContext;
+import edu.umd.cs.findbugs.annotations.NonNull;
+import java.net.InetSocketAddress;
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.stream.Collectors;
+import org.apache.commons.net.util.SubnetUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This translator returns the proxy address of the private subnet containing the Cassandra node IP,
+ * or default address if no matching subnets, or passes through the original node address if no
+ * default configured.
+ *
+ * <p>The translator can be used for scenarios when all nodes are behind some kind of proxy, and
+ * that proxy is different for nodes located in different subnets (eg. when Cassandra is deployed in
+ * multiple datacenters/regions). One can use this, for example, for Cassandra on Kubernetes with
+ * different Cassandra datacenters deployed to different Kubernetes clusters.
+ */
+public class SubnetAddressTranslator implements AddressTranslator {
+
+ private static final Logger LOG = LoggerFactory.getLogger(SubnetAddressTranslator.class);
+
+ /**
+ * A comma separated list with Cassandra node subnets (CIDR notations) to target addresses in a
+ * format `<subnet>:<hostname>:<port>`, for example:
+ * `100.64.0.0/15:cassandra.datacenter1.com:9042,100.66.0.0/15:cassandra.datacenter1.com:9042`. If
+ * configured without port, the default 9042 will be used.
+ */
+ public static final String ADDRESS_TRANSLATOR_SUBNET_ADDRESSES =
+ "advanced.address-translator.subnet-addresses";
+ /**
+ * A default address to fallback to if Cassandra node IP isn't contained in any of the configured
+ * subnets. If configured without port, the default 9042 will be used.
+ */
+ public static final String ADDRESS_TRANSLATOR_DEFAULT_ADDRESS =
+ "advanced.address-translator.default-address";
+
+ public static DriverOption ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_SUBNET_ADDRESSES;
+ }
+ };
+ public static DriverOption ADDRESS_TRANSLATOR_DEFAULT_ADDRESS_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_DEFAULT_ADDRESS;
+ }
+ };
+
+ private static final String DELIMITER = ":";
+ private static final int DEFAULT_PORT = 9042;
+
+ private final List<SubnetAddress> subnetAddresses;
+ private final Optional<InetSocketAddress> defaultAddress;
+ private final String logPrefix;
+
+ public SubnetAddressTranslator(@NonNull DriverContext context) {
+ logPrefix = context.getSessionName();
+ this.subnetAddresses =
+ context.getConfig().getDefaultProfile()
+ .getStringList(ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION).stream()
Review Comment:
From what I can tell this would work this way:
```hocon
advanced.address-translator
{
subnet-addresses = ["100.64.0.0/15:cassandra.datacenter1.com:9042","100.66.0.0/15:cassandra.datacenter1.com:9042"]
}
```
One concern I have is placing multiple property values in the same string, which implies some kind of parsing. I suppose that was already the case with address:port, but would be nice to make this structured. Using `:` as a delimiter would also cause some problems with IPv6.
could we consider using a string map instead, e.g.:
```hocon
advanced.address-translator
{
subnet-addresses {
100.64.0.0/15 = "cassandra.datacenter1.com:9042"
100.66.0.0/15 = "cassandra.datacenter1.com:9042"
}
}
```
although i suspect this might break parsing some how with a `/` in a key name, but I think doing something like this could make the config more human/machine readable.
##########
core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/SubnetAddressTranslator.java:
##########
@@ -0,0 +1,210 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.datastax.oss.driver.internal.core.addresstranslation;
+
+import com.datastax.oss.driver.api.core.addresstranslation.AddressTranslator;
+import com.datastax.oss.driver.api.core.config.DriverOption;
+import com.datastax.oss.driver.api.core.context.DriverContext;
+import edu.umd.cs.findbugs.annotations.NonNull;
+import java.net.InetSocketAddress;
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.stream.Collectors;
+import org.apache.commons.net.util.SubnetUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This translator returns the proxy address of the private subnet containing the Cassandra node IP,
+ * or default address if no matching subnets, or passes through the original node address if no
+ * default configured.
+ *
+ * <p>The translator can be used for scenarios when all nodes are behind some kind of proxy, and
+ * that proxy is different for nodes located in different subnets (eg. when Cassandra is deployed in
+ * multiple datacenters/regions). One can use this, for example, for Cassandra on Kubernetes with
+ * different Cassandra datacenters deployed to different Kubernetes clusters.
+ */
+public class SubnetAddressTranslator implements AddressTranslator {
+
+ private static final Logger LOG = LoggerFactory.getLogger(SubnetAddressTranslator.class);
+
+ /**
+ * A comma separated list with Cassandra node subnets (CIDR notations) to target addresses in a
+ * format `<subnet>:<hostname>:<port>`, for example:
+ * `100.64.0.0/15:cassandra.datacenter1.com:9042,100.66.0.0/15:cassandra.datacenter1.com:9042`. If
Review Comment:
This expects a list, not a comma separated string right?
##########
core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/SubnetAddressTranslator.java:
##########
@@ -0,0 +1,210 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.datastax.oss.driver.internal.core.addresstranslation;
+
+import com.datastax.oss.driver.api.core.addresstranslation.AddressTranslator;
+import com.datastax.oss.driver.api.core.config.DriverOption;
+import com.datastax.oss.driver.api.core.context.DriverContext;
+import edu.umd.cs.findbugs.annotations.NonNull;
+import java.net.InetSocketAddress;
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.stream.Collectors;
+import org.apache.commons.net.util.SubnetUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This translator returns the proxy address of the private subnet containing the Cassandra node IP,
+ * or default address if no matching subnets, or passes through the original node address if no
+ * default configured.
+ *
+ * <p>The translator can be used for scenarios when all nodes are behind some kind of proxy, and
+ * that proxy is different for nodes located in different subnets (eg. when Cassandra is deployed in
+ * multiple datacenters/regions). One can use this, for example, for Cassandra on Kubernetes with
+ * different Cassandra datacenters deployed to different Kubernetes clusters.
+ */
+public class SubnetAddressTranslator implements AddressTranslator {
+
+ private static final Logger LOG = LoggerFactory.getLogger(SubnetAddressTranslator.class);
+
+ /**
+ * A comma separated list with Cassandra node subnets (CIDR notations) to target addresses in a
+ * format `<subnet>:<hostname>:<port>`, for example:
+ * `100.64.0.0/15:cassandra.datacenter1.com:9042,100.66.0.0/15:cassandra.datacenter1.com:9042`. If
+ * configured without port, the default 9042 will be used.
+ */
+ public static final String ADDRESS_TRANSLATOR_SUBNET_ADDRESSES =
+ "advanced.address-translator.subnet-addresses";
+ /**
+ * A default address to fallback to if Cassandra node IP isn't contained in any of the configured
+ * subnets. If configured without port, the default 9042 will be used.
+ */
+ public static final String ADDRESS_TRANSLATOR_DEFAULT_ADDRESS =
+ "advanced.address-translator.default-address";
Review Comment:
this might be considered property overloading, but wonder if we should just fallback on `advertised-hostname` that `FixedHostNameAddressTranslator` uses?
##########
core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/SubnetAddressTranslator.java:
##########
@@ -0,0 +1,210 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.datastax.oss.driver.internal.core.addresstranslation;
+
+import com.datastax.oss.driver.api.core.addresstranslation.AddressTranslator;
+import com.datastax.oss.driver.api.core.config.DriverOption;
+import com.datastax.oss.driver.api.core.context.DriverContext;
+import edu.umd.cs.findbugs.annotations.NonNull;
+import java.net.InetSocketAddress;
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.stream.Collectors;
+import org.apache.commons.net.util.SubnetUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This translator returns the proxy address of the private subnet containing the Cassandra node IP,
+ * or default address if no matching subnets, or passes through the original node address if no
+ * default configured.
+ *
+ * <p>The translator can be used for scenarios when all nodes are behind some kind of proxy, and
+ * that proxy is different for nodes located in different subnets (eg. when Cassandra is deployed in
+ * multiple datacenters/regions). One can use this, for example, for Cassandra on Kubernetes with
+ * different Cassandra datacenters deployed to different Kubernetes clusters.
+ */
+public class SubnetAddressTranslator implements AddressTranslator {
+
+ private static final Logger LOG = LoggerFactory.getLogger(SubnetAddressTranslator.class);
+
+ /**
+ * A comma separated list with Cassandra node subnets (CIDR notations) to target addresses in a
+ * format `<subnet>:<hostname>:<port>`, for example:
+ * `100.64.0.0/15:cassandra.datacenter1.com:9042,100.66.0.0/15:cassandra.datacenter1.com:9042`. If
+ * configured without port, the default 9042 will be used.
+ */
+ public static final String ADDRESS_TRANSLATOR_SUBNET_ADDRESSES =
+ "advanced.address-translator.subnet-addresses";
+ /**
+ * A default address to fallback to if Cassandra node IP isn't contained in any of the configured
+ * subnets. If configured without port, the default 9042 will be used.
+ */
+ public static final String ADDRESS_TRANSLATOR_DEFAULT_ADDRESS =
+ "advanced.address-translator.default-address";
+
+ public static DriverOption ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_SUBNET_ADDRESSES;
+ }
+ };
+ public static DriverOption ADDRESS_TRANSLATOR_DEFAULT_ADDRESS_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_DEFAULT_ADDRESS;
+ }
+ };
+
+ private static final String DELIMITER = ":";
+ private static final int DEFAULT_PORT = 9042;
+
+ private final List<SubnetAddress> subnetAddresses;
+ private final Optional<InetSocketAddress> defaultAddress;
+ private final String logPrefix;
+
+ public SubnetAddressTranslator(@NonNull DriverContext context) {
+ logPrefix = context.getSessionName();
+ this.subnetAddresses =
+ context.getConfig().getDefaultProfile()
+ .getStringList(ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION).stream()
+ .map(SubnetAddress::fromString)
+ .collect(Collectors.toList());
+ this.defaultAddress =
+ Optional.ofNullable(
+ context
+ .getConfig()
+ .getDefaultProfile()
+ .getString(ADDRESS_TRANSLATOR_DEFAULT_ADDRESS_OPTION, null))
+ .map(SubnetAddressTranslator::parseAddress);
+ SubnetAddressTranslator.validateSubnetsAreNotOverlapping(this.subnetAddresses);
+ }
+
+ @NonNull
+ @Override
+ public InetSocketAddress translate(@NonNull InetSocketAddress address) {
+ InetSocketAddress translatedAddress = null;
+ for (SubnetAddress subnetAddress : subnetAddresses) {
+ if (subnetAddress.contains(address)) {
+ translatedAddress = subnetAddress.address;
+ }
+ }
+ if (translatedAddress == null && defaultAddress.isPresent()) {
+ translatedAddress = defaultAddress.get();
+ }
+ if (translatedAddress == null) {
+ translatedAddress = address;
+ }
+ LOG.debug("[{}] Resolved {} to {}", logPrefix, address, translatedAddress);
+ return translatedAddress;
+ }
+
+ @Override
+ public void close() {}
+
+ private static InetSocketAddress parseAddress(String address) {
+ String[] addressTuple = address.split(DELIMITER);
Review Comment:
Noticed that the two uses of `split` and `getClass` in equals causes some error prone warnings (that would prevent maven compiling):
```
[WARNING] /home/tolbertam/Documents/Projects/cassandra-java-driver/core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/SubnetAddressTranslator.java:[126,42] [StringSplitter] String.split(String) has surprising behavior
(see https://errorprone.info/bugpattern/StringSplitter)
Did you mean 'List<String> addressTuple = Splitter.onPattern(DELIMITER).splitToList(address);'?
[WARNING] /home/tolbertam/Documents/Projects/cassandra-java-driver/core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/SubnetAddressTranslator.java:[177,20] [EqualsGetClass] Overriding Object#equals in a non-final class by using getClass rather than instanceof breaks substitutability of subclasses.
(see https://errorprone.info/bugpattern/EqualsGetClass)
Did you mean 'if (!(other instanceof SubnetAddress)) return false;'?
[WARNING] /home/tolbertam/Documents/Projects/cassandra-java-driver/core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/SubnetAddressTranslator.java:[199,56] [StringSplitter] String.split(String) has surprising behavior
(see https://errorprone.info/bugpattern/StringSplitter)
Did you mean 'List<String> subnetAddressTuple = Splitter.onPattern(DELIMITER).splitToList(subnetAddress);'?
```
##########
core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/SubnetAddressTranslator.java:
##########
@@ -0,0 +1,210 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.datastax.oss.driver.internal.core.addresstranslation;
+
+import com.datastax.oss.driver.api.core.addresstranslation.AddressTranslator;
+import com.datastax.oss.driver.api.core.config.DriverOption;
+import com.datastax.oss.driver.api.core.context.DriverContext;
+import edu.umd.cs.findbugs.annotations.NonNull;
+import java.net.InetSocketAddress;
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.stream.Collectors;
+import org.apache.commons.net.util.SubnetUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This translator returns the proxy address of the private subnet containing the Cassandra node IP,
+ * or default address if no matching subnets, or passes through the original node address if no
+ * default configured.
+ *
+ * <p>The translator can be used for scenarios when all nodes are behind some kind of proxy, and
+ * that proxy is different for nodes located in different subnets (eg. when Cassandra is deployed in
+ * multiple datacenters/regions). One can use this, for example, for Cassandra on Kubernetes with
+ * different Cassandra datacenters deployed to different Kubernetes clusters.
+ */
+public class SubnetAddressTranslator implements AddressTranslator {
+
+ private static final Logger LOG = LoggerFactory.getLogger(SubnetAddressTranslator.class);
+
+ /**
+ * A comma separated list with Cassandra node subnets (CIDR notations) to target addresses in a
+ * format `<subnet>:<hostname>:<port>`, for example:
+ * `100.64.0.0/15:cassandra.datacenter1.com:9042,100.66.0.0/15:cassandra.datacenter1.com:9042`. If
+ * configured without port, the default 9042 will be used.
+ */
+ public static final String ADDRESS_TRANSLATOR_SUBNET_ADDRESSES =
+ "advanced.address-translator.subnet-addresses";
+ /**
+ * A default address to fallback to if Cassandra node IP isn't contained in any of the configured
+ * subnets. If configured without port, the default 9042 will be used.
+ */
+ public static final String ADDRESS_TRANSLATOR_DEFAULT_ADDRESS =
+ "advanced.address-translator.default-address";
+
+ public static DriverOption ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_SUBNET_ADDRESSES;
+ }
+ };
+ public static DriverOption ADDRESS_TRANSLATOR_DEFAULT_ADDRESS_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_DEFAULT_ADDRESS;
+ }
+ };
+
+ private static final String DELIMITER = ":";
+ private static final int DEFAULT_PORT = 9042;
+
+ private final List<SubnetAddress> subnetAddresses;
+ private final Optional<InetSocketAddress> defaultAddress;
+ private final String logPrefix;
+
+ public SubnetAddressTranslator(@NonNull DriverContext context) {
+ logPrefix = context.getSessionName();
+ this.subnetAddresses =
+ context.getConfig().getDefaultProfile()
+ .getStringList(ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION).stream()
+ .map(SubnetAddress::fromString)
+ .collect(Collectors.toList());
+ this.defaultAddress =
+ Optional.ofNullable(
+ context
+ .getConfig()
+ .getDefaultProfile()
+ .getString(ADDRESS_TRANSLATOR_DEFAULT_ADDRESS_OPTION, null))
+ .map(SubnetAddressTranslator::parseAddress);
+ SubnetAddressTranslator.validateSubnetsAreNotOverlapping(this.subnetAddresses);
+ }
+
+ @NonNull
+ @Override
+ public InetSocketAddress translate(@NonNull InetSocketAddress address) {
+ InetSocketAddress translatedAddress = null;
+ for (SubnetAddress subnetAddress : subnetAddresses) {
+ if (subnetAddress.contains(address)) {
+ translatedAddress = subnetAddress.address;
+ }
+ }
+ if (translatedAddress == null && defaultAddress.isPresent()) {
+ translatedAddress = defaultAddress.get();
+ }
+ if (translatedAddress == null) {
+ translatedAddress = address;
+ }
+ LOG.debug("[{}] Resolved {} to {}", logPrefix, address, translatedAddress);
+ return translatedAddress;
+ }
+
+ @Override
+ public void close() {}
+
+ private static InetSocketAddress parseAddress(String address) {
+ String[] addressTuple = address.split(DELIMITER);
+ if (addressTuple.length == 2) {
+ return new InetSocketAddress(addressTuple[0], Integer.parseInt(addressTuple[1]));
+ }
+ if (addressTuple.length == 1) {
+ return new InetSocketAddress(addressTuple[0], DEFAULT_PORT);
+ }
+ throw new IllegalArgumentException("Invalid default address: " + address);
+ }
+
+ private static void validateSubnetsAreNotOverlapping(List<SubnetAddress> subnetAddresses) {
+ for (int i = 0; i < subnetAddresses.size() - 1; i++) {
Review Comment:
This is good to have :+1:. I imagine without this, there could be some unpredictable behavior if subnets overlap (unclear which endpoint would ultimately be used).
##########
core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/SubnetAddressTranslator.java:
##########
@@ -0,0 +1,210 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.datastax.oss.driver.internal.core.addresstranslation;
+
+import com.datastax.oss.driver.api.core.addresstranslation.AddressTranslator;
+import com.datastax.oss.driver.api.core.config.DriverOption;
+import com.datastax.oss.driver.api.core.context.DriverContext;
+import edu.umd.cs.findbugs.annotations.NonNull;
+import java.net.InetSocketAddress;
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.stream.Collectors;
+import org.apache.commons.net.util.SubnetUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This translator returns the proxy address of the private subnet containing the Cassandra node IP,
+ * or default address if no matching subnets, or passes through the original node address if no
+ * default configured.
+ *
+ * <p>The translator can be used for scenarios when all nodes are behind some kind of proxy, and
+ * that proxy is different for nodes located in different subnets (eg. when Cassandra is deployed in
+ * multiple datacenters/regions). One can use this, for example, for Cassandra on Kubernetes with
+ * different Cassandra datacenters deployed to different Kubernetes clusters.
+ */
+public class SubnetAddressTranslator implements AddressTranslator {
+
+ private static final Logger LOG = LoggerFactory.getLogger(SubnetAddressTranslator.class);
+
+ /**
+ * A comma separated list with Cassandra node subnets (CIDR notations) to target addresses in a
+ * format `<subnet>:<hostname>:<port>`, for example:
+ * `100.64.0.0/15:cassandra.datacenter1.com:9042,100.66.0.0/15:cassandra.datacenter1.com:9042`. If
Review Comment:
Should also use `cassandra.datacenter2.com:9042` for the second entry just to disambiguate the two configs.
##########
core/src/main/resources/reference.conf:
##########
@@ -1020,8 +1020,9 @@ datastax-java-driver {
# the package com.datastax.oss.driver.internal.core.addresstranslation.
#
# The driver provides the following implementations out of the box:
- # - PassThroughAddressTranslator: returns all addresses unchanged
+ # - PassThroughAddressTranslator: returns all addresses unchanged.
Review Comment:
In regards to your question around additional documentation needed, it would be nice to add something documenting `SubnetAddressTranslator` to the [Address resolution docs](https://github.com/apache/cassandra-java-driver/tree/4.x/manual/core/address_resolution) below the `EC2 Multi region` section. Could be good to explain the use case for why this may be considered. If you are feeling adventurous it'd be nice to add a blurb on `FixedHostNameAddressTranslator` as well (I could also make an attempt at this later)
##########
core/pom.xml:
##########
@@ -116,6 +116,10 @@
<groupId>org.reactivestreams</groupId>
<artifactId>reactive-streams</artifactId>
</dependency>
+ <dependency>
+ <groupId>commons-net</groupId>
+ <artifactId>commons-net</artifactId>
+ </dependency>
Review Comment:
I think if we add any dependencies for this, we should make them optional, as I expect folks will commonly not use an address translator, so we should instead have users who desire to use this feature add this as a dependency in their project (which we could document the requirement for somewhere).
Also I think it would be nice to consider using https://github.com/seancfoley/IPAddress instead, as [C* also uses this](https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/config/SubnetGroups.java) and it carries the benefits of having no other dependencies and it also supports IPv6.
##########
core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/SubnetAddressTranslator.java:
##########
@@ -0,0 +1,210 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.datastax.oss.driver.internal.core.addresstranslation;
+
+import com.datastax.oss.driver.api.core.addresstranslation.AddressTranslator;
+import com.datastax.oss.driver.api.core.config.DriverOption;
+import com.datastax.oss.driver.api.core.context.DriverContext;
+import edu.umd.cs.findbugs.annotations.NonNull;
+import java.net.InetSocketAddress;
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.stream.Collectors;
+import org.apache.commons.net.util.SubnetUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This translator returns the proxy address of the private subnet containing the Cassandra node IP,
+ * or default address if no matching subnets, or passes through the original node address if no
+ * default configured.
+ *
+ * <p>The translator can be used for scenarios when all nodes are behind some kind of proxy, and
+ * that proxy is different for nodes located in different subnets (eg. when Cassandra is deployed in
+ * multiple datacenters/regions). One can use this, for example, for Cassandra on Kubernetes with
+ * different Cassandra datacenters deployed to different Kubernetes clusters.
+ */
+public class SubnetAddressTranslator implements AddressTranslator {
+
+ private static final Logger LOG = LoggerFactory.getLogger(SubnetAddressTranslator.class);
+
+ /**
+ * A comma separated list with Cassandra node subnets (CIDR notations) to target addresses in a
+ * format `<subnet>:<hostname>:<port>`, for example:
+ * `100.64.0.0/15:cassandra.datacenter1.com:9042,100.66.0.0/15:cassandra.datacenter1.com:9042`. If
+ * configured without port, the default 9042 will be used.
+ */
+ public static final String ADDRESS_TRANSLATOR_SUBNET_ADDRESSES =
+ "advanced.address-translator.subnet-addresses";
+ /**
Review Comment:
small nit: add newline between field and the next field doc.
##########
core/src/main/resources/reference.conf:
##########
@@ -1020,8 +1020,9 @@ datastax-java-driver {
# the package com.datastax.oss.driver.internal.core.addresstranslation.
#
# The driver provides the following implementations out of the box:
- # - PassThroughAddressTranslator: returns all addresses unchanged
+ # - PassThroughAddressTranslator: returns all addresses unchanged.
# - FixedHostNameAddressTranslator: translates all addresses to a specific hostname.
+ # - SubnetAddressTranslator: translates addresses to hostname based on the subnet matches.
Review Comment:
Like `advertised-hostname` below, we should also add a commented out example of configuring `subnet-addresses` for `SubnetAddressTranslator`
##########
core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/SubnetAddressTranslator.java:
##########
@@ -0,0 +1,210 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.datastax.oss.driver.internal.core.addresstranslation;
+
+import com.datastax.oss.driver.api.core.addresstranslation.AddressTranslator;
+import com.datastax.oss.driver.api.core.config.DriverOption;
+import com.datastax.oss.driver.api.core.context.DriverContext;
+import edu.umd.cs.findbugs.annotations.NonNull;
+import java.net.InetSocketAddress;
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.stream.Collectors;
+import org.apache.commons.net.util.SubnetUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This translator returns the proxy address of the private subnet containing the Cassandra node IP,
+ * or default address if no matching subnets, or passes through the original node address if no
+ * default configured.
+ *
+ * <p>The translator can be used for scenarios when all nodes are behind some kind of proxy, and
+ * that proxy is different for nodes located in different subnets (eg. when Cassandra is deployed in
+ * multiple datacenters/regions). One can use this, for example, for Cassandra on Kubernetes with
+ * different Cassandra datacenters deployed to different Kubernetes clusters.
+ */
+public class SubnetAddressTranslator implements AddressTranslator {
+
+ private static final Logger LOG = LoggerFactory.getLogger(SubnetAddressTranslator.class);
+
+ /**
+ * A comma separated list with Cassandra node subnets (CIDR notations) to target addresses in a
+ * format `<subnet>:<hostname>:<port>`, for example:
+ * `100.64.0.0/15:cassandra.datacenter1.com:9042,100.66.0.0/15:cassandra.datacenter1.com:9042`. If
+ * configured without port, the default 9042 will be used.
+ */
+ public static final String ADDRESS_TRANSLATOR_SUBNET_ADDRESSES =
+ "advanced.address-translator.subnet-addresses";
+ /**
+ * A default address to fallback to if Cassandra node IP isn't contained in any of the configured
+ * subnets. If configured without port, the default 9042 will be used.
+ */
+ public static final String ADDRESS_TRANSLATOR_DEFAULT_ADDRESS =
+ "advanced.address-translator.default-address";
+
+ public static DriverOption ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_SUBNET_ADDRESSES;
+ }
+ };
+ public static DriverOption ADDRESS_TRANSLATOR_DEFAULT_ADDRESS_OPTION =
Review Comment:
small nit: add newline between field and the next field when multi-line.
##########
core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/SubnetAddressTranslator.java:
##########
@@ -0,0 +1,210 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.datastax.oss.driver.internal.core.addresstranslation;
+
+import com.datastax.oss.driver.api.core.addresstranslation.AddressTranslator;
+import com.datastax.oss.driver.api.core.config.DriverOption;
+import com.datastax.oss.driver.api.core.context.DriverContext;
+import edu.umd.cs.findbugs.annotations.NonNull;
+import java.net.InetSocketAddress;
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.stream.Collectors;
+import org.apache.commons.net.util.SubnetUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This translator returns the proxy address of the private subnet containing the Cassandra node IP,
+ * or default address if no matching subnets, or passes through the original node address if no
+ * default configured.
+ *
+ * <p>The translator can be used for scenarios when all nodes are behind some kind of proxy, and
+ * that proxy is different for nodes located in different subnets (eg. when Cassandra is deployed in
+ * multiple datacenters/regions). One can use this, for example, for Cassandra on Kubernetes with
+ * different Cassandra datacenters deployed to different Kubernetes clusters.
+ */
+public class SubnetAddressTranslator implements AddressTranslator {
+
+ private static final Logger LOG = LoggerFactory.getLogger(SubnetAddressTranslator.class);
+
+ /**
+ * A comma separated list with Cassandra node subnets (CIDR notations) to target addresses in a
+ * format `<subnet>:<hostname>:<port>`, for example:
+ * `100.64.0.0/15:cassandra.datacenter1.com:9042,100.66.0.0/15:cassandra.datacenter1.com:9042`. If
+ * configured without port, the default 9042 will be used.
+ */
+ public static final String ADDRESS_TRANSLATOR_SUBNET_ADDRESSES =
+ "advanced.address-translator.subnet-addresses";
+ /**
+ * A default address to fallback to if Cassandra node IP isn't contained in any of the configured
+ * subnets. If configured without port, the default 9042 will be used.
+ */
+ public static final String ADDRESS_TRANSLATOR_DEFAULT_ADDRESS =
+ "advanced.address-translator.default-address";
Review Comment:
Actually, I just realized `FixedHostNameAddressTranslator` currently does not offer the capability of providing a port, where this implementation does (either you can or can't). So I do think perhaps having a separate property is ok. Maybe we should revisit the capability in `FixedHostNameAddressTranslator` later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "jahstreet (via GitHub)" <gi...@apache.org>.
jahstreet commented on PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#issuecomment-2961365667
Sorry to make you spend much time time on it. Rebase is obviously a good thing to have always. On it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator to translate Cassandra node IPs from private network based on its subnet mask [cassandra-java-driver]
Posted by "jahstreet (via GitHub)" <gi...@apache.org>.
jahstreet commented on PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#issuecomment-2961586771
Did the visual check and corrected guava references. Hope now it is in a good shape 🤞 .
@absurdfarce how (with which maven args) do you build/compile locally? I wasn't able to reproduce the dependency errors with basic `mvn clean package -DskipTests`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator to translate Cassandra node IPs from private network based on its subnet mask [cassandra-java-driver]
Posted by "jahstreet (via GitHub)" <gi...@apache.org>.
jahstreet commented on PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#issuecomment-2970883719
Thank you folks for guiding me 🙏 .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#discussion_r1956490115
##########
core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/SubnetAddressTranslator.java:
##########
@@ -0,0 +1,226 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.datastax.oss.driver.internal.core.addresstranslation;
+
+import com.datastax.oss.driver.api.core.addresstranslation.AddressTranslator;
+import com.datastax.oss.driver.api.core.config.DriverOption;
+import com.datastax.oss.driver.api.core.context.DriverContext;
+import com.google.common.base.Splitter;
+import edu.umd.cs.findbugs.annotations.NonNull;
+import inet.ipaddr.IPAddress;
+import inet.ipaddr.IPAddressString;
+import java.net.InetSocketAddress;
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.stream.Collectors;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This translator returns the proxy address of the private subnet containing the Cassandra node IP,
+ * or default address if no matching subnets, or passes through the original node address if no
+ * default configured.
+ *
+ * <p>The translator can be used for scenarios when all nodes are behind some kind of proxy, and
+ * that proxy is different for nodes located in different subnets (eg. when Cassandra is deployed in
+ * multiple datacenters/regions). One can use this, for example, for Cassandra on Kubernetes with
+ * different Cassandra datacenters deployed to different Kubernetes clusters.
+ */
+public class SubnetAddressTranslator implements AddressTranslator {
+
+ private static final Logger LOG = LoggerFactory.getLogger(SubnetAddressTranslator.class);
+
+ /**
+ * A map of Cassandra node subnets (CIDR notations) to target addresses, for example (note quoted
+ * keys):
+ *
+ * <pre>
+ * advanced.address-translator.subnet-addresses {
+ * "100.64.0.0/15" = "cassandra.datacenter1.com:9042"
+ * "100.66.0.0/15" = "cassandra.datacenter2.com"
+ * # IPv6 example:
+ * # "::ffff:6440:0/111" = "cassandra.datacenter1.com:9042"
+ * # "::ffff:6442:0/111" = "cassandra.datacenter2.com"
+ * }
+ * </pre>
+ *
+ * If configured without port, the default 9042 will be used. Also supports IPv6 subnets. Note:
+ * subnets must be represented as prefix blocks, see {@link inet.ipaddr.Address#isPrefixBlock()}.
+ */
+ public static final String ADDRESS_TRANSLATOR_SUBNET_ADDRESSES =
+ "advanced.address-translator.subnet-addresses";
+
+ /**
+ * A default address to fallback to if Cassandra node IP isn't contained in any of the configured
+ * subnets. If configured without port, the default 9042 will be used. Also supports IPv6
+ * addresses.
+ */
+ public static final String ADDRESS_TRANSLATOR_DEFAULT_ADDRESS =
+ "advanced.address-translator.default-address";
+
+ public static DriverOption ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_SUBNET_ADDRESSES;
+ }
+ };
+
+ public static DriverOption ADDRESS_TRANSLATOR_DEFAULT_ADDRESS_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_DEFAULT_ADDRESS;
+ }
+ };
+
+ private static final String DELIMITER = ":";
+ private static final int DEFAULT_PORT = 9042;
+
+ private final List<SubnetAddress> subnetAddresses;
+ private final Optional<InetSocketAddress> defaultAddress;
+ private final String logPrefix;
+
+ public SubnetAddressTranslator(@NonNull DriverContext context) {
+ logPrefix = context.getSessionName();
+ this.subnetAddresses =
+ context.getConfig().getDefaultProfile()
+ .getStringMap(ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION).entrySet().stream()
+ .map(
+ e -> {
+ // Quoted and/or containing forward slashes map keys in reference.conf are read to
+ // strings with additional quotes, eg. 100.64.0.0/15 -> '100.64.0."0/15"' or
+ // "100.64.0.0/15" -> '"100.64.0.0/15"'
+ String subnet = e.getKey().replaceAll("\"", "");
+ String address = e.getValue();
+ return new SubnetAddress(subnet, address);
+ })
+ .collect(Collectors.toList());
+ this.defaultAddress =
+ Optional.ofNullable(
+ context
+ .getConfig()
+ .getDefaultProfile()
+ .getString(ADDRESS_TRANSLATOR_DEFAULT_ADDRESS_OPTION, null))
+ .map(SubnetAddressTranslator::parseAddress);
+ SubnetAddressTranslator.validateSubnetsAreNotOverlapping(this.subnetAddresses);
+ }
+
+ @NonNull
+ @Override
+ public InetSocketAddress translate(@NonNull InetSocketAddress address) {
+ InetSocketAddress translatedAddress = null;
+ for (SubnetAddress subnetAddress : subnetAddresses) {
+ if (subnetAddress.contains(address)) {
+ translatedAddress = subnetAddress.address;
+ }
+ }
+ if (translatedAddress == null && defaultAddress.isPresent()) {
+ translatedAddress = defaultAddress.get();
+ }
+ if (translatedAddress == null) {
+ translatedAddress = address;
+ }
+ LOG.debug("[{}] Resolved {} to {}", logPrefix, address, translatedAddress);
+ return translatedAddress;
+ }
+
+ @Override
+ public void close() {}
+
+ private static InetSocketAddress parseAddress(String address) {
+ List<String> addressTuple = Splitter.onPattern(DELIMITER).splitToList(address);
+ if (addressTuple.size() == 2) {
+ return new InetSocketAddress(addressTuple.get(0), Integer.parseInt(addressTuple.get(1)));
Review Comment:
👍 the way it exists now look good to me. Will give this another review pass and likely +1 it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "jahstreet (via GitHub)" <gi...@apache.org>.
jahstreet commented on code in PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#discussion_r2000549859
##########
core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/SubnetAddressTranslator.java:
##########
@@ -0,0 +1,238 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.datastax.oss.driver.internal.core.addresstranslation;
+
+import com.datastax.oss.driver.api.core.addresstranslation.AddressTranslator;
+import com.datastax.oss.driver.api.core.config.DriverOption;
+import com.datastax.oss.driver.api.core.context.DriverContext;
+import com.datastax.oss.driver.internal.core.util.AddressUtils;
+import edu.umd.cs.findbugs.annotations.NonNull;
+import edu.umd.cs.findbugs.annotations.Nullable;
+import inet.ipaddr.IPAddress;
+import inet.ipaddr.IPAddressString;
+import java.net.InetSocketAddress;
+import java.util.List;
+import java.util.Optional;
+import java.util.stream.Collectors;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This translator returns the proxy address of the private subnet containing the Cassandra node IP,
+ * or default address if no matching subnets, or passes through the original node address if no
+ * default configured.
+ *
+ * <p>The translator can be used for scenarios when all nodes are behind some kind of proxy, and
+ * that proxy is different for nodes located in different subnets (eg. when Cassandra is deployed in
+ * multiple datacenters/regions). One can use this, for example, for Cassandra on Kubernetes with
+ * different Cassandra datacenters deployed to different Kubernetes clusters.
+ */
+public class SubnetAddressTranslator implements AddressTranslator {
+ private static final Logger LOG = LoggerFactory.getLogger(SubnetAddressTranslator.class);
+
+ /**
+ * A map of Cassandra node subnets (CIDR notations) to target addresses, for example (note quoted
+ * keys):
+ *
+ * <pre>
+ * advanced.address-translator.subnet-addresses {
+ * "100.64.0.0/15" = "cassandra.datacenter1.com:9042"
+ * "100.66.0.0/15" = "cassandra.datacenter2.com:9042"
+ * # IPv6 example:
+ * # "::ffff:6440:0/111" = "cassandra.datacenter1.com:9042"
+ * # "::ffff:6442:0/111" = "cassandra.datacenter2.com:9042"
+ * }
+ * </pre>
+ *
+ * Note: subnets must be represented as prefix blocks, see {@link
+ * inet.ipaddr.Address#isPrefixBlock()}.
+ */
+ public static final String ADDRESS_TRANSLATOR_SUBNET_ADDRESSES =
+ "advanced.address-translator.subnet-addresses";
+
+ /**
+ * A default address to fallback to if Cassandra node IP isn't contained in any of the configured
+ * subnets.
+ */
+ public static final String ADDRESS_TRANSLATOR_DEFAULT_ADDRESS =
+ "advanced.address-translator.default-address";
+
+ /**
+ * Whether to resolve the addresses on initialization (if true) or on each node (re-)connection
+ * (if false). Defaults to false.
+ */
+ public static final String ADDRESS_TRANSLATOR_RESOLVE_ADDRESSES =
+ "advanced.address-translator.resolve-addresses";
+
+ public static DriverOption ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_SUBNET_ADDRESSES;
+ }
+ };
+
+ public static DriverOption ADDRESS_TRANSLATOR_DEFAULT_ADDRESS_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_DEFAULT_ADDRESS;
+ }
+ };
+
+ public static DriverOption ADDRESS_TRANSLATOR_RESOLVE_ADDRESSES_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_RESOLVE_ADDRESSES;
+ }
+ };
Review Comment:
Indeed, that was derived from the example.
I can move them, sure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator to translate Cassandra node IPs from private network based on its subnet mask [cassandra-java-driver]
Posted by "absurdfarce (via GitHub)" <gi...@apache.org>.
absurdfarce commented on PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#issuecomment-2963932249
These changes look good @jahstreet , thanks for jumping on this so fast! A local test build seemed to succeed without a problem so I've kicked off a Jenkins build to (a) confirm that we're good and (b) throw all the unit and integration tests at it. I'm cautiously optimistic we've got it this time!
As for what I've been running... most of the time I've been using the same [command](https://github.com/apache/cassandra-java-driver/blob/4.19.0/Jenkinsfile-datastax#L121) used by the DataStax Jenkinsfile (`mvn -B -V install -DskipTests -Dmaven.javadoc.skip=true`) but I've been able to reproduce it with the even simpler `mvn clean install -DskipTests=true`. Note that I did _not_ see this failure in my local test builds until I manually added the change to make guava-shaded an optional dependency; prior to that change the only place I saw it was on Jenkins. And I'm pretty sure that's because Jenkins was merging the changes in the PR into the current state of 4.x... which meant the commit making the guava-shaded dependency optional was brought into play. That's just a hypothesis but it seems to fit all the facts.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "jahstreet (via GitHub)" <gi...@apache.org>.
jahstreet commented on PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#issuecomment-2952368535
Deal, thx for the feedback.
Will address it asap.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "absurdfarce (via GitHub)" <gi...@apache.org>.
absurdfarce commented on code in PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#discussion_r2127386825
##########
core/src/test/java/com/datastax/oss/driver/internal/core/addresstranslation/SubnetTest.java:
##########
@@ -0,0 +1,101 @@
+package com.datastax.oss.driver.internal.core.addresstranslation;
Review Comment:
Last one: need ASF license text at the top of this source file
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "jahstreet (via GitHub)" <gi...@apache.org>.
jahstreet commented on code in PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#discussion_r1950480009
##########
core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/SubnetAddressTranslator.java:
##########
@@ -0,0 +1,210 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.datastax.oss.driver.internal.core.addresstranslation;
+
+import com.datastax.oss.driver.api.core.addresstranslation.AddressTranslator;
+import com.datastax.oss.driver.api.core.config.DriverOption;
+import com.datastax.oss.driver.api.core.context.DriverContext;
+import edu.umd.cs.findbugs.annotations.NonNull;
+import java.net.InetSocketAddress;
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.stream.Collectors;
+import org.apache.commons.net.util.SubnetUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This translator returns the proxy address of the private subnet containing the Cassandra node IP,
+ * or default address if no matching subnets, or passes through the original node address if no
+ * default configured.
+ *
+ * <p>The translator can be used for scenarios when all nodes are behind some kind of proxy, and
+ * that proxy is different for nodes located in different subnets (eg. when Cassandra is deployed in
+ * multiple datacenters/regions). One can use this, for example, for Cassandra on Kubernetes with
+ * different Cassandra datacenters deployed to different Kubernetes clusters.
+ */
+public class SubnetAddressTranslator implements AddressTranslator {
+
+ private static final Logger LOG = LoggerFactory.getLogger(SubnetAddressTranslator.class);
+
+ /**
+ * A comma separated list with Cassandra node subnets (CIDR notations) to target addresses in a
+ * format `<subnet>:<hostname>:<port>`, for example:
+ * `100.64.0.0/15:cassandra.datacenter1.com:9042,100.66.0.0/15:cassandra.datacenter1.com:9042`. If
+ * configured without port, the default 9042 will be used.
+ */
+ public static final String ADDRESS_TRANSLATOR_SUBNET_ADDRESSES =
+ "advanced.address-translator.subnet-addresses";
+ /**
+ * A default address to fallback to if Cassandra node IP isn't contained in any of the configured
+ * subnets. If configured without port, the default 9042 will be used.
+ */
+ public static final String ADDRESS_TRANSLATOR_DEFAULT_ADDRESS =
+ "advanced.address-translator.default-address";
+
+ public static DriverOption ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_SUBNET_ADDRESSES;
+ }
+ };
+ public static DriverOption ADDRESS_TRANSLATOR_DEFAULT_ADDRESS_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_DEFAULT_ADDRESS;
+ }
+ };
+
+ private static final String DELIMITER = ":";
+ private static final int DEFAULT_PORT = 9042;
+
+ private final List<SubnetAddress> subnetAddresses;
+ private final Optional<InetSocketAddress> defaultAddress;
+ private final String logPrefix;
+
+ public SubnetAddressTranslator(@NonNull DriverContext context) {
+ logPrefix = context.getSessionName();
+ this.subnetAddresses =
+ context.getConfig().getDefaultProfile()
+ .getStringList(ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION).stream()
Review Comment:
You are right about the list vs string, I've got it wrong from Spark Cassandra Connector.
WRT to a map - looks cool, lemme try it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "jahstreet (via GitHub)" <gi...@apache.org>.
jahstreet commented on code in PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#discussion_r1955680480
##########
core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/SubnetAddressTranslator.java:
##########
@@ -0,0 +1,226 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.datastax.oss.driver.internal.core.addresstranslation;
+
+import com.datastax.oss.driver.api.core.addresstranslation.AddressTranslator;
+import com.datastax.oss.driver.api.core.config.DriverOption;
+import com.datastax.oss.driver.api.core.context.DriverContext;
+import com.google.common.base.Splitter;
+import edu.umd.cs.findbugs.annotations.NonNull;
+import inet.ipaddr.IPAddress;
+import inet.ipaddr.IPAddressString;
+import java.net.InetSocketAddress;
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.stream.Collectors;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This translator returns the proxy address of the private subnet containing the Cassandra node IP,
+ * or default address if no matching subnets, or passes through the original node address if no
+ * default configured.
+ *
+ * <p>The translator can be used for scenarios when all nodes are behind some kind of proxy, and
+ * that proxy is different for nodes located in different subnets (eg. when Cassandra is deployed in
+ * multiple datacenters/regions). One can use this, for example, for Cassandra on Kubernetes with
+ * different Cassandra datacenters deployed to different Kubernetes clusters.
+ */
+public class SubnetAddressTranslator implements AddressTranslator {
+
+ private static final Logger LOG = LoggerFactory.getLogger(SubnetAddressTranslator.class);
+
+ /**
+ * A map of Cassandra node subnets (CIDR notations) to target addresses, for example (note quoted
+ * keys):
+ *
+ * <pre>
+ * advanced.address-translator.subnet-addresses {
+ * "100.64.0.0/15" = "cassandra.datacenter1.com:9042"
+ * "100.66.0.0/15" = "cassandra.datacenter2.com"
+ * # IPv6 example:
+ * # "::ffff:6440:0/111" = "cassandra.datacenter1.com:9042"
+ * # "::ffff:6442:0/111" = "cassandra.datacenter2.com"
+ * }
+ * </pre>
+ *
+ * If configured without port, the default 9042 will be used. Also supports IPv6 subnets. Note:
+ * subnets must be represented as prefix blocks, see {@link inet.ipaddr.Address#isPrefixBlock()}.
+ */
+ public static final String ADDRESS_TRANSLATOR_SUBNET_ADDRESSES =
+ "advanced.address-translator.subnet-addresses";
+
+ /**
+ * A default address to fallback to if Cassandra node IP isn't contained in any of the configured
+ * subnets. If configured without port, the default 9042 will be used. Also supports IPv6
+ * addresses.
+ */
+ public static final String ADDRESS_TRANSLATOR_DEFAULT_ADDRESS =
+ "advanced.address-translator.default-address";
+
+ public static DriverOption ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_SUBNET_ADDRESSES;
+ }
+ };
+
+ public static DriverOption ADDRESS_TRANSLATOR_DEFAULT_ADDRESS_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_DEFAULT_ADDRESS;
+ }
+ };
+
+ private static final String DELIMITER = ":";
+ private static final int DEFAULT_PORT = 9042;
+
+ private final List<SubnetAddress> subnetAddresses;
+ private final Optional<InetSocketAddress> defaultAddress;
+ private final String logPrefix;
+
+ public SubnetAddressTranslator(@NonNull DriverContext context) {
+ logPrefix = context.getSessionName();
+ this.subnetAddresses =
+ context.getConfig().getDefaultProfile()
+ .getStringMap(ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION).entrySet().stream()
+ .map(
+ e -> {
+ // Quoted and/or containing forward slashes map keys in reference.conf are read to
+ // strings with additional quotes, eg. 100.64.0.0/15 -> '100.64.0."0/15"' or
+ // "100.64.0.0/15" -> '"100.64.0.0/15"'
+ String subnet = e.getKey().replaceAll("\"", "");
+ String address = e.getValue();
+ return new SubnetAddress(subnet, address);
+ })
+ .collect(Collectors.toList());
+ this.defaultAddress =
+ Optional.ofNullable(
+ context
+ .getConfig()
+ .getDefaultProfile()
+ .getString(ADDRESS_TRANSLATOR_DEFAULT_ADDRESS_OPTION, null))
+ .map(SubnetAddressTranslator::parseAddress);
+ SubnetAddressTranslator.validateSubnetsAreNotOverlapping(this.subnetAddresses);
+ }
+
+ @NonNull
+ @Override
+ public InetSocketAddress translate(@NonNull InetSocketAddress address) {
+ InetSocketAddress translatedAddress = null;
+ for (SubnetAddress subnetAddress : subnetAddresses) {
+ if (subnetAddress.contains(address)) {
+ translatedAddress = subnetAddress.address;
+ }
+ }
+ if (translatedAddress == null && defaultAddress.isPresent()) {
+ translatedAddress = defaultAddress.get();
+ }
+ if (translatedAddress == null) {
+ translatedAddress = address;
+ }
+ LOG.debug("[{}] Resolved {} to {}", logPrefix, address, translatedAddress);
+ return translatedAddress;
+ }
+
+ @Override
+ public void close() {}
+
+ private static InetSocketAddress parseAddress(String address) {
+ List<String> addressTuple = Splitter.onPattern(DELIMITER).splitToList(address);
+ if (addressTuple.size() == 2) {
+ return new InetSocketAddress(addressTuple.get(0), Integer.parseInt(addressTuple.get(1)));
Review Comment:
Got ya. Not much left to tweak, so good we got to this point. Nice conversation 👍 . Picking it up!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator to translate Cassandra node IPs from private network based on its subnet mask [cassandra-java-driver]
Posted by "absurdfarce (via GitHub)" <gi...@apache.org>.
absurdfarce merged PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "jahstreet (via GitHub)" <gi...@apache.org>.
jahstreet commented on code in PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#discussion_r1954021486
##########
core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/SubnetAddressTranslator.java:
##########
@@ -0,0 +1,226 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.datastax.oss.driver.internal.core.addresstranslation;
+
+import com.datastax.oss.driver.api.core.addresstranslation.AddressTranslator;
+import com.datastax.oss.driver.api.core.config.DriverOption;
+import com.datastax.oss.driver.api.core.context.DriverContext;
+import com.google.common.base.Splitter;
+import edu.umd.cs.findbugs.annotations.NonNull;
+import inet.ipaddr.IPAddress;
+import inet.ipaddr.IPAddressString;
+import java.net.InetSocketAddress;
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.stream.Collectors;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This translator returns the proxy address of the private subnet containing the Cassandra node IP,
+ * or default address if no matching subnets, or passes through the original node address if no
+ * default configured.
+ *
+ * <p>The translator can be used for scenarios when all nodes are behind some kind of proxy, and
+ * that proxy is different for nodes located in different subnets (eg. when Cassandra is deployed in
+ * multiple datacenters/regions). One can use this, for example, for Cassandra on Kubernetes with
+ * different Cassandra datacenters deployed to different Kubernetes clusters.
+ */
+public class SubnetAddressTranslator implements AddressTranslator {
+
+ private static final Logger LOG = LoggerFactory.getLogger(SubnetAddressTranslator.class);
+
+ /**
+ * A map of Cassandra node subnets (CIDR notations) to target addresses, for example (note quoted
+ * keys):
+ *
+ * <pre>
+ * advanced.address-translator.subnet-addresses {
+ * "100.64.0.0/15" = "cassandra.datacenter1.com:9042"
+ * "100.66.0.0/15" = "cassandra.datacenter2.com"
+ * # IPv6 example:
+ * # "::ffff:6440:0/111" = "cassandra.datacenter1.com:9042"
+ * # "::ffff:6442:0/111" = "cassandra.datacenter2.com"
+ * }
+ * </pre>
+ *
+ * If configured without port, the default 9042 will be used. Also supports IPv6 subnets. Note:
+ * subnets must be represented as prefix blocks, see {@link inet.ipaddr.Address#isPrefixBlock()}.
+ */
+ public static final String ADDRESS_TRANSLATOR_SUBNET_ADDRESSES =
+ "advanced.address-translator.subnet-addresses";
+
+ /**
+ * A default address to fallback to if Cassandra node IP isn't contained in any of the configured
+ * subnets. If configured without port, the default 9042 will be used. Also supports IPv6
+ * addresses.
+ */
+ public static final String ADDRESS_TRANSLATOR_DEFAULT_ADDRESS =
+ "advanced.address-translator.default-address";
+
+ public static DriverOption ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_SUBNET_ADDRESSES;
+ }
+ };
+
+ public static DriverOption ADDRESS_TRANSLATOR_DEFAULT_ADDRESS_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_DEFAULT_ADDRESS;
+ }
+ };
+
+ private static final String DELIMITER = ":";
+ private static final int DEFAULT_PORT = 9042;
+
+ private final List<SubnetAddress> subnetAddresses;
+ private final Optional<InetSocketAddress> defaultAddress;
+ private final String logPrefix;
+
+ public SubnetAddressTranslator(@NonNull DriverContext context) {
+ logPrefix = context.getSessionName();
+ this.subnetAddresses =
+ context.getConfig().getDefaultProfile()
+ .getStringMap(ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION).entrySet().stream()
+ .map(
+ e -> {
+ // Quoted and/or containing forward slashes map keys in reference.conf are read to
+ // strings with additional quotes, eg. 100.64.0.0/15 -> '100.64.0."0/15"' or
+ // "100.64.0.0/15" -> '"100.64.0.0/15"'
+ String subnet = e.getKey().replaceAll("\"", "");
+ String address = e.getValue();
+ return new SubnetAddress(subnet, address);
+ })
+ .collect(Collectors.toList());
+ this.defaultAddress =
+ Optional.ofNullable(
+ context
+ .getConfig()
+ .getDefaultProfile()
+ .getString(ADDRESS_TRANSLATOR_DEFAULT_ADDRESS_OPTION, null))
+ .map(SubnetAddressTranslator::parseAddress);
+ SubnetAddressTranslator.validateSubnetsAreNotOverlapping(this.subnetAddresses);
+ }
+
+ @NonNull
+ @Override
+ public InetSocketAddress translate(@NonNull InetSocketAddress address) {
+ InetSocketAddress translatedAddress = null;
+ for (SubnetAddress subnetAddress : subnetAddresses) {
+ if (subnetAddress.contains(address)) {
+ translatedAddress = subnetAddress.address;
+ }
+ }
+ if (translatedAddress == null && defaultAddress.isPresent()) {
+ translatedAddress = defaultAddress.get();
+ }
+ if (translatedAddress == null) {
+ translatedAddress = address;
+ }
+ LOG.debug("[{}] Resolved {} to {}", logPrefix, address, translatedAddress);
+ return translatedAddress;
+ }
+
+ @Override
+ public void close() {}
+
+ private static InetSocketAddress parseAddress(String address) {
+ List<String> addressTuple = Splitter.onPattern(DELIMITER).splitToList(address);
+ if (addressTuple.size() == 2) {
+ return new InetSocketAddress(addressTuple.get(0), Integer.parseInt(addressTuple.get(1)));
Review Comment:
I also have a mixed feeling of overloading `advanced.resolve-contact-points` property. It asks for renaming if we reuse it here. WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "jahstreet (via GitHub)" <gi...@apache.org>.
jahstreet commented on code in PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#discussion_r1948873334
##########
core/pom.xml:
##########
@@ -116,6 +116,10 @@
<groupId>org.reactivestreams</groupId>
<artifactId>reactive-streams</artifactId>
</dependency>
+ <dependency>
+ <groupId>commons-net</groupId>
+ <artifactId>commons-net</artifactId>
+ </dependency>
Review Comment:
What is the strategy around adding dependencies? Is that fine to have it here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "jahstreet (via GitHub)" <gi...@apache.org>.
jahstreet commented on PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#issuecomment-2661018616
> Excellent work @jahstreet, thank you! Have some suggestions, but I'm +1 either way.
Thanks! I will push a commit with annotations till Monday morning.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "jahstreet (via GitHub)" <gi...@apache.org>.
jahstreet commented on code in PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#discussion_r2000568199
##########
core/src/main/java/com/datastax/oss/driver/internal/core/ContactPoints.java:
##########
@@ -41,7 +38,22 @@ public static Set<EndPoint> merge(
Set<EndPoint> result = Sets.newHashSet(programmaticContactPoints);
for (String spec : configContactPoints) {
- for (InetSocketAddress address : extract(spec, resolve)) {
+
+ Set<InetSocketAddress> addresses = Collections.emptySet();
+ try {
+ addresses = AddressUtils.extract(spec, resolve);
+ } catch (RuntimeException e) {
+ LOG.warn("Ignoring invalid contact point {} ({})", spec, e.getMessage(), e);
+ }
Review Comment:
Previously, `#extract` code was used only in this class and we logged errors together with reasons of these errors. Now the info about reasons is moved to util method, which is called from multiple places. In this class, I aimed to keep logging (as well as other functionality) as close to the origin as seemed possible to avoid opinionated refactoring, so I needed a way to get reasons of errors from the utility `#extract` to log them together with the context logs.
Happy to agree on the way it should look like and change accordingly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator to translate Cassandra node IPs from private network based on its subnet mask [cassandra-java-driver]
Posted by "jahstreet (via GitHub)" <gi...@apache.org>.
jahstreet commented on PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#issuecomment-2964475860
Thx for the hints @absurdfarce 🦅 👁️ . Indeed, the fast refactoring outputted the tiny bug. Thx for checking.
🤞 for another CI run.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "absurdfarce (via GitHub)" <gi...@apache.org>.
absurdfarce commented on code in PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#discussion_r2127386442
##########
core/src/test/java/com/datastax/oss/driver/internal/core/addresstranslation/SubnetAddressTest.java:
##########
@@ -0,0 +1,27 @@
+package com.datastax.oss.driver.internal.core.addresstranslation;
Review Comment:
Need ASF license text at the top of this source file here as well
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator to translate Cassandra node IPs from private network based on its subnet mask [cassandra-java-driver]
Posted by "absurdfarce (via GitHub)" <gi...@apache.org>.
absurdfarce commented on PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#issuecomment-2971184944
Thanks @tolbertam . And a huge thank you to @jahstreet for your persistence on this one!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "jahstreet (via GitHub)" <gi...@apache.org>.
jahstreet commented on code in PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#discussion_r1954075719
##########
core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/SubnetAddressTranslator.java:
##########
@@ -0,0 +1,226 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.datastax.oss.driver.internal.core.addresstranslation;
+
+import com.datastax.oss.driver.api.core.addresstranslation.AddressTranslator;
+import com.datastax.oss.driver.api.core.config.DriverOption;
+import com.datastax.oss.driver.api.core.context.DriverContext;
+import com.google.common.base.Splitter;
+import edu.umd.cs.findbugs.annotations.NonNull;
+import inet.ipaddr.IPAddress;
+import inet.ipaddr.IPAddressString;
+import java.net.InetSocketAddress;
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.stream.Collectors;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This translator returns the proxy address of the private subnet containing the Cassandra node IP,
+ * or default address if no matching subnets, or passes through the original node address if no
+ * default configured.
+ *
+ * <p>The translator can be used for scenarios when all nodes are behind some kind of proxy, and
+ * that proxy is different for nodes located in different subnets (eg. when Cassandra is deployed in
+ * multiple datacenters/regions). One can use this, for example, for Cassandra on Kubernetes with
+ * different Cassandra datacenters deployed to different Kubernetes clusters.
+ */
+public class SubnetAddressTranslator implements AddressTranslator {
+
+ private static final Logger LOG = LoggerFactory.getLogger(SubnetAddressTranslator.class);
+
+ /**
+ * A map of Cassandra node subnets (CIDR notations) to target addresses, for example (note quoted
+ * keys):
+ *
+ * <pre>
+ * advanced.address-translator.subnet-addresses {
+ * "100.64.0.0/15" = "cassandra.datacenter1.com:9042"
+ * "100.66.0.0/15" = "cassandra.datacenter2.com"
+ * # IPv6 example:
+ * # "::ffff:6440:0/111" = "cassandra.datacenter1.com:9042"
+ * # "::ffff:6442:0/111" = "cassandra.datacenter2.com"
+ * }
+ * </pre>
+ *
+ * If configured without port, the default 9042 will be used. Also supports IPv6 subnets. Note:
+ * subnets must be represented as prefix blocks, see {@link inet.ipaddr.Address#isPrefixBlock()}.
+ */
+ public static final String ADDRESS_TRANSLATOR_SUBNET_ADDRESSES =
+ "advanced.address-translator.subnet-addresses";
+
+ /**
+ * A default address to fallback to if Cassandra node IP isn't contained in any of the configured
+ * subnets. If configured without port, the default 9042 will be used. Also supports IPv6
+ * addresses.
+ */
+ public static final String ADDRESS_TRANSLATOR_DEFAULT_ADDRESS =
+ "advanced.address-translator.default-address";
+
+ public static DriverOption ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_SUBNET_ADDRESSES;
+ }
+ };
+
+ public static DriverOption ADDRESS_TRANSLATOR_DEFAULT_ADDRESS_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_DEFAULT_ADDRESS;
+ }
+ };
+
+ private static final String DELIMITER = ":";
+ private static final int DEFAULT_PORT = 9042;
+
+ private final List<SubnetAddress> subnetAddresses;
+ private final Optional<InetSocketAddress> defaultAddress;
+ private final String logPrefix;
+
+ public SubnetAddressTranslator(@NonNull DriverContext context) {
+ logPrefix = context.getSessionName();
+ this.subnetAddresses =
+ context.getConfig().getDefaultProfile()
+ .getStringMap(ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION).entrySet().stream()
+ .map(
+ e -> {
+ // Quoted and/or containing forward slashes map keys in reference.conf are read to
+ // strings with additional quotes, eg. 100.64.0.0/15 -> '100.64.0."0/15"' or
+ // "100.64.0.0/15" -> '"100.64.0.0/15"'
+ String subnet = e.getKey().replaceAll("\"", "");
+ String address = e.getValue();
+ return new SubnetAddress(subnet, address);
+ })
+ .collect(Collectors.toList());
+ this.defaultAddress =
+ Optional.ofNullable(
+ context
+ .getConfig()
+ .getDefaultProfile()
+ .getString(ADDRESS_TRANSLATOR_DEFAULT_ADDRESS_OPTION, null))
+ .map(SubnetAddressTranslator::parseAddress);
+ SubnetAddressTranslator.validateSubnetsAreNotOverlapping(this.subnetAddresses);
+ }
+
+ @NonNull
+ @Override
+ public InetSocketAddress translate(@NonNull InetSocketAddress address) {
+ InetSocketAddress translatedAddress = null;
+ for (SubnetAddress subnetAddress : subnetAddresses) {
+ if (subnetAddress.contains(address)) {
+ translatedAddress = subnetAddress.address;
+ }
+ }
+ if (translatedAddress == null && defaultAddress.isPresent()) {
+ translatedAddress = defaultAddress.get();
+ }
+ if (translatedAddress == null) {
+ translatedAddress = address;
+ }
+ LOG.debug("[{}] Resolved {} to {}", logPrefix, address, translatedAddress);
+ return translatedAddress;
+ }
+
+ @Override
+ public void close() {}
+
+ private static InetSocketAddress parseAddress(String address) {
+ List<String> addressTuple = Splitter.onPattern(DELIMITER).splitToList(address);
+ if (addressTuple.size() == 2) {
+ return new InetSocketAddress(addressTuple.get(0), Integer.parseInt(addressTuple.get(1)));
Review Comment:
I also see some logging happening in `ContactPoints#extract` which might become confusing on debugging. So it asks for refactoring if we wanna reuse it. How about moving it to util package?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "jahstreet (via GitHub)" <gi...@apache.org>.
jahstreet commented on code in PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#discussion_r1956557403
##########
core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/SubnetAddressTranslator.java:
##########
@@ -0,0 +1,226 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.datastax.oss.driver.internal.core.addresstranslation;
+
+import com.datastax.oss.driver.api.core.addresstranslation.AddressTranslator;
+import com.datastax.oss.driver.api.core.config.DriverOption;
+import com.datastax.oss.driver.api.core.context.DriverContext;
+import com.google.common.base.Splitter;
+import edu.umd.cs.findbugs.annotations.NonNull;
+import inet.ipaddr.IPAddress;
+import inet.ipaddr.IPAddressString;
+import java.net.InetSocketAddress;
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.stream.Collectors;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This translator returns the proxy address of the private subnet containing the Cassandra node IP,
+ * or default address if no matching subnets, or passes through the original node address if no
+ * default configured.
+ *
+ * <p>The translator can be used for scenarios when all nodes are behind some kind of proxy, and
+ * that proxy is different for nodes located in different subnets (eg. when Cassandra is deployed in
+ * multiple datacenters/regions). One can use this, for example, for Cassandra on Kubernetes with
+ * different Cassandra datacenters deployed to different Kubernetes clusters.
+ */
+public class SubnetAddressTranslator implements AddressTranslator {
+
+ private static final Logger LOG = LoggerFactory.getLogger(SubnetAddressTranslator.class);
+
+ /**
+ * A map of Cassandra node subnets (CIDR notations) to target addresses, for example (note quoted
+ * keys):
+ *
+ * <pre>
+ * advanced.address-translator.subnet-addresses {
+ * "100.64.0.0/15" = "cassandra.datacenter1.com:9042"
+ * "100.66.0.0/15" = "cassandra.datacenter2.com"
+ * # IPv6 example:
+ * # "::ffff:6440:0/111" = "cassandra.datacenter1.com:9042"
+ * # "::ffff:6442:0/111" = "cassandra.datacenter2.com"
+ * }
+ * </pre>
+ *
+ * If configured without port, the default 9042 will be used. Also supports IPv6 subnets. Note:
+ * subnets must be represented as prefix blocks, see {@link inet.ipaddr.Address#isPrefixBlock()}.
+ */
+ public static final String ADDRESS_TRANSLATOR_SUBNET_ADDRESSES =
+ "advanced.address-translator.subnet-addresses";
+
+ /**
+ * A default address to fallback to if Cassandra node IP isn't contained in any of the configured
+ * subnets. If configured without port, the default 9042 will be used. Also supports IPv6
+ * addresses.
+ */
+ public static final String ADDRESS_TRANSLATOR_DEFAULT_ADDRESS =
+ "advanced.address-translator.default-address";
+
+ public static DriverOption ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_SUBNET_ADDRESSES;
+ }
+ };
+
+ public static DriverOption ADDRESS_TRANSLATOR_DEFAULT_ADDRESS_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_DEFAULT_ADDRESS;
+ }
+ };
+
+ private static final String DELIMITER = ":";
+ private static final int DEFAULT_PORT = 9042;
+
+ private final List<SubnetAddress> subnetAddresses;
+ private final Optional<InetSocketAddress> defaultAddress;
+ private final String logPrefix;
+
+ public SubnetAddressTranslator(@NonNull DriverContext context) {
+ logPrefix = context.getSessionName();
+ this.subnetAddresses =
+ context.getConfig().getDefaultProfile()
+ .getStringMap(ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION).entrySet().stream()
+ .map(
+ e -> {
+ // Quoted and/or containing forward slashes map keys in reference.conf are read to
+ // strings with additional quotes, eg. 100.64.0.0/15 -> '100.64.0."0/15"' or
+ // "100.64.0.0/15" -> '"100.64.0.0/15"'
+ String subnet = e.getKey().replaceAll("\"", "");
+ String address = e.getValue();
+ return new SubnetAddress(subnet, address);
+ })
+ .collect(Collectors.toList());
+ this.defaultAddress =
+ Optional.ofNullable(
+ context
+ .getConfig()
+ .getDefaultProfile()
+ .getString(ADDRESS_TRANSLATOR_DEFAULT_ADDRESS_OPTION, null))
+ .map(SubnetAddressTranslator::parseAddress);
+ SubnetAddressTranslator.validateSubnetsAreNotOverlapping(this.subnetAddresses);
+ }
+
+ @NonNull
+ @Override
+ public InetSocketAddress translate(@NonNull InetSocketAddress address) {
+ InetSocketAddress translatedAddress = null;
+ for (SubnetAddress subnetAddress : subnetAddresses) {
+ if (subnetAddress.contains(address)) {
+ translatedAddress = subnetAddress.address;
+ }
+ }
+ if (translatedAddress == null && defaultAddress.isPresent()) {
+ translatedAddress = defaultAddress.get();
+ }
+ if (translatedAddress == null) {
+ translatedAddress = address;
+ }
+ LOG.debug("[{}] Resolved {} to {}", logPrefix, address, translatedAddress);
+ return translatedAddress;
+ }
+
+ @Override
+ public void close() {}
+
+ private static InetSocketAddress parseAddress(String address) {
+ List<String> addressTuple = Splitter.onPattern(DELIMITER).splitToList(address);
+ if (addressTuple.size() == 2) {
+ return new InetSocketAddress(addressTuple.get(0), Integer.parseInt(addressTuple.get(1)));
Review Comment:
thank you Andy 🤝
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "jahstreet (via GitHub)" <gi...@apache.org>.
jahstreet commented on code in PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#discussion_r1954257461
##########
core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/SubnetAddressTranslator.java:
##########
@@ -0,0 +1,226 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.datastax.oss.driver.internal.core.addresstranslation;
+
+import com.datastax.oss.driver.api.core.addresstranslation.AddressTranslator;
+import com.datastax.oss.driver.api.core.config.DriverOption;
+import com.datastax.oss.driver.api.core.context.DriverContext;
+import com.google.common.base.Splitter;
+import edu.umd.cs.findbugs.annotations.NonNull;
+import inet.ipaddr.IPAddress;
+import inet.ipaddr.IPAddressString;
+import java.net.InetSocketAddress;
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.stream.Collectors;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This translator returns the proxy address of the private subnet containing the Cassandra node IP,
+ * or default address if no matching subnets, or passes through the original node address if no
+ * default configured.
+ *
+ * <p>The translator can be used for scenarios when all nodes are behind some kind of proxy, and
+ * that proxy is different for nodes located in different subnets (eg. when Cassandra is deployed in
+ * multiple datacenters/regions). One can use this, for example, for Cassandra on Kubernetes with
+ * different Cassandra datacenters deployed to different Kubernetes clusters.
+ */
+public class SubnetAddressTranslator implements AddressTranslator {
+
+ private static final Logger LOG = LoggerFactory.getLogger(SubnetAddressTranslator.class);
+
+ /**
+ * A map of Cassandra node subnets (CIDR notations) to target addresses, for example (note quoted
+ * keys):
+ *
+ * <pre>
+ * advanced.address-translator.subnet-addresses {
+ * "100.64.0.0/15" = "cassandra.datacenter1.com:9042"
+ * "100.66.0.0/15" = "cassandra.datacenter2.com"
+ * # IPv6 example:
+ * # "::ffff:6440:0/111" = "cassandra.datacenter1.com:9042"
+ * # "::ffff:6442:0/111" = "cassandra.datacenter2.com"
+ * }
+ * </pre>
+ *
+ * If configured without port, the default 9042 will be used. Also supports IPv6 subnets. Note:
+ * subnets must be represented as prefix blocks, see {@link inet.ipaddr.Address#isPrefixBlock()}.
+ */
+ public static final String ADDRESS_TRANSLATOR_SUBNET_ADDRESSES =
+ "advanced.address-translator.subnet-addresses";
+
+ /**
+ * A default address to fallback to if Cassandra node IP isn't contained in any of the configured
+ * subnets. If configured without port, the default 9042 will be used. Also supports IPv6
+ * addresses.
+ */
+ public static final String ADDRESS_TRANSLATOR_DEFAULT_ADDRESS =
+ "advanced.address-translator.default-address";
+
+ public static DriverOption ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_SUBNET_ADDRESSES;
+ }
+ };
+
+ public static DriverOption ADDRESS_TRANSLATOR_DEFAULT_ADDRESS_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_DEFAULT_ADDRESS;
+ }
+ };
+
+ private static final String DELIMITER = ":";
+ private static final int DEFAULT_PORT = 9042;
+
+ private final List<SubnetAddress> subnetAddresses;
+ private final Optional<InetSocketAddress> defaultAddress;
+ private final String logPrefix;
+
+ public SubnetAddressTranslator(@NonNull DriverContext context) {
+ logPrefix = context.getSessionName();
+ this.subnetAddresses =
+ context.getConfig().getDefaultProfile()
+ .getStringMap(ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION).entrySet().stream()
+ .map(
+ e -> {
+ // Quoted and/or containing forward slashes map keys in reference.conf are read to
+ // strings with additional quotes, eg. 100.64.0.0/15 -> '100.64.0."0/15"' or
+ // "100.64.0.0/15" -> '"100.64.0.0/15"'
+ String subnet = e.getKey().replaceAll("\"", "");
+ String address = e.getValue();
+ return new SubnetAddress(subnet, address);
+ })
+ .collect(Collectors.toList());
+ this.defaultAddress =
+ Optional.ofNullable(
+ context
+ .getConfig()
+ .getDefaultProfile()
+ .getString(ADDRESS_TRANSLATOR_DEFAULT_ADDRESS_OPTION, null))
+ .map(SubnetAddressTranslator::parseAddress);
+ SubnetAddressTranslator.validateSubnetsAreNotOverlapping(this.subnetAddresses);
+ }
+
+ @NonNull
+ @Override
+ public InetSocketAddress translate(@NonNull InetSocketAddress address) {
+ InetSocketAddress translatedAddress = null;
+ for (SubnetAddress subnetAddress : subnetAddresses) {
+ if (subnetAddress.contains(address)) {
+ translatedAddress = subnetAddress.address;
+ }
+ }
+ if (translatedAddress == null && defaultAddress.isPresent()) {
+ translatedAddress = defaultAddress.get();
+ }
+ if (translatedAddress == null) {
+ translatedAddress = address;
+ }
+ LOG.debug("[{}] Resolved {} to {}", logPrefix, address, translatedAddress);
+ return translatedAddress;
+ }
+
+ @Override
+ public void close() {}
+
+ private static InetSocketAddress parseAddress(String address) {
+ List<String> addressTuple = Splitter.onPattern(DELIMITER).splitToList(address);
+ if (addressTuple.size() == 2) {
+ return new InetSocketAddress(addressTuple.get(0), Integer.parseInt(addressTuple.get(1)));
Review Comment:
Refactored by moving it the new AddressUtils.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "jahstreet (via GitHub)" <gi...@apache.org>.
jahstreet commented on PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#issuecomment-2960385965
Suggested commit message:
"Add SubnetAddressTranslator to translate Cassandra node IPs from private network based on its subnet mask"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "absurdfarce (via GitHub)" <gi...@apache.org>.
absurdfarce commented on PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#issuecomment-2961383755
No worries @jahstreet ... I'm just happy we figured it out and have a clear path now to get this in!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator to translate Cassandra node IPs from private network based on its subnet mask [cassandra-java-driver]
Posted by "jahstreet (via GitHub)" <gi...@apache.org>.
jahstreet commented on code in PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#discussion_r2139431858
##########
core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/Subnet.java:
##########
@@ -0,0 +1,176 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.datastax.oss.driver.internal.core.addresstranslation;
+
+import com.datastax.oss.driver.shaded.guava.common.annotations.VisibleForTesting;
+import com.google.common.base.Splitter;
Review Comment:
```suggestion
com.datastax.oss.driver.shaded.guava.common.base.Splitter;
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "jahstreet (via GitHub)" <gi...@apache.org>.
jahstreet commented on code in PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#discussion_r1953986294
##########
core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/SubnetAddressTranslator.java:
##########
@@ -0,0 +1,226 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.datastax.oss.driver.internal.core.addresstranslation;
+
+import com.datastax.oss.driver.api.core.addresstranslation.AddressTranslator;
+import com.datastax.oss.driver.api.core.config.DriverOption;
+import com.datastax.oss.driver.api.core.context.DriverContext;
+import com.google.common.base.Splitter;
+import edu.umd.cs.findbugs.annotations.NonNull;
+import inet.ipaddr.IPAddress;
+import inet.ipaddr.IPAddressString;
+import java.net.InetSocketAddress;
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.stream.Collectors;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This translator returns the proxy address of the private subnet containing the Cassandra node IP,
+ * or default address if no matching subnets, or passes through the original node address if no
+ * default configured.
+ *
+ * <p>The translator can be used for scenarios when all nodes are behind some kind of proxy, and
+ * that proxy is different for nodes located in different subnets (eg. when Cassandra is deployed in
+ * multiple datacenters/regions). One can use this, for example, for Cassandra on Kubernetes with
+ * different Cassandra datacenters deployed to different Kubernetes clusters.
+ */
+public class SubnetAddressTranslator implements AddressTranslator {
+
+ private static final Logger LOG = LoggerFactory.getLogger(SubnetAddressTranslator.class);
+
+ /**
+ * A map of Cassandra node subnets (CIDR notations) to target addresses, for example (note quoted
+ * keys):
+ *
+ * <pre>
+ * advanced.address-translator.subnet-addresses {
+ * "100.64.0.0/15" = "cassandra.datacenter1.com:9042"
+ * "100.66.0.0/15" = "cassandra.datacenter2.com"
+ * # IPv6 example:
+ * # "::ffff:6440:0/111" = "cassandra.datacenter1.com:9042"
+ * # "::ffff:6442:0/111" = "cassandra.datacenter2.com"
+ * }
+ * </pre>
+ *
+ * If configured without port, the default 9042 will be used. Also supports IPv6 subnets. Note:
+ * subnets must be represented as prefix blocks, see {@link inet.ipaddr.Address#isPrefixBlock()}.
+ */
+ public static final String ADDRESS_TRANSLATOR_SUBNET_ADDRESSES =
+ "advanced.address-translator.subnet-addresses";
+
+ /**
+ * A default address to fallback to if Cassandra node IP isn't contained in any of the configured
+ * subnets. If configured without port, the default 9042 will be used. Also supports IPv6
Review Comment:
Okay, agree.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "jahstreet (via GitHub)" <gi...@apache.org>.
jahstreet commented on code in PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#discussion_r1955722161
##########
core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/SubnetAddressTranslator.java:
##########
@@ -0,0 +1,226 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.datastax.oss.driver.internal.core.addresstranslation;
+
+import com.datastax.oss.driver.api.core.addresstranslation.AddressTranslator;
+import com.datastax.oss.driver.api.core.config.DriverOption;
+import com.datastax.oss.driver.api.core.context.DriverContext;
+import com.google.common.base.Splitter;
+import edu.umd.cs.findbugs.annotations.NonNull;
+import inet.ipaddr.IPAddress;
+import inet.ipaddr.IPAddressString;
+import java.net.InetSocketAddress;
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.stream.Collectors;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This translator returns the proxy address of the private subnet containing the Cassandra node IP,
+ * or default address if no matching subnets, or passes through the original node address if no
+ * default configured.
+ *
+ * <p>The translator can be used for scenarios when all nodes are behind some kind of proxy, and
+ * that proxy is different for nodes located in different subnets (eg. when Cassandra is deployed in
+ * multiple datacenters/regions). One can use this, for example, for Cassandra on Kubernetes with
+ * different Cassandra datacenters deployed to different Kubernetes clusters.
+ */
+public class SubnetAddressTranslator implements AddressTranslator {
+
+ private static final Logger LOG = LoggerFactory.getLogger(SubnetAddressTranslator.class);
+
+ /**
+ * A map of Cassandra node subnets (CIDR notations) to target addresses, for example (note quoted
+ * keys):
+ *
+ * <pre>
+ * advanced.address-translator.subnet-addresses {
+ * "100.64.0.0/15" = "cassandra.datacenter1.com:9042"
+ * "100.66.0.0/15" = "cassandra.datacenter2.com"
+ * # IPv6 example:
+ * # "::ffff:6440:0/111" = "cassandra.datacenter1.com:9042"
+ * # "::ffff:6442:0/111" = "cassandra.datacenter2.com"
+ * }
+ * </pre>
+ *
+ * If configured without port, the default 9042 will be used. Also supports IPv6 subnets. Note:
+ * subnets must be represented as prefix blocks, see {@link inet.ipaddr.Address#isPrefixBlock()}.
+ */
+ public static final String ADDRESS_TRANSLATOR_SUBNET_ADDRESSES =
+ "advanced.address-translator.subnet-addresses";
+
+ /**
+ * A default address to fallback to if Cassandra node IP isn't contained in any of the configured
+ * subnets. If configured without port, the default 9042 will be used. Also supports IPv6
+ * addresses.
+ */
+ public static final String ADDRESS_TRANSLATOR_DEFAULT_ADDRESS =
+ "advanced.address-translator.default-address";
+
+ public static DriverOption ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_SUBNET_ADDRESSES;
+ }
+ };
+
+ public static DriverOption ADDRESS_TRANSLATOR_DEFAULT_ADDRESS_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_DEFAULT_ADDRESS;
+ }
+ };
+
+ private static final String DELIMITER = ":";
+ private static final int DEFAULT_PORT = 9042;
+
+ private final List<SubnetAddress> subnetAddresses;
+ private final Optional<InetSocketAddress> defaultAddress;
+ private final String logPrefix;
+
+ public SubnetAddressTranslator(@NonNull DriverContext context) {
+ logPrefix = context.getSessionName();
+ this.subnetAddresses =
+ context.getConfig().getDefaultProfile()
+ .getStringMap(ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION).entrySet().stream()
+ .map(
+ e -> {
+ // Quoted and/or containing forward slashes map keys in reference.conf are read to
+ // strings with additional quotes, eg. 100.64.0.0/15 -> '100.64.0."0/15"' or
+ // "100.64.0.0/15" -> '"100.64.0.0/15"'
+ String subnet = e.getKey().replaceAll("\"", "");
+ String address = e.getValue();
+ return new SubnetAddress(subnet, address);
+ })
+ .collect(Collectors.toList());
+ this.defaultAddress =
+ Optional.ofNullable(
+ context
+ .getConfig()
+ .getDefaultProfile()
+ .getString(ADDRESS_TRANSLATOR_DEFAULT_ADDRESS_OPTION, null))
+ .map(SubnetAddressTranslator::parseAddress);
+ SubnetAddressTranslator.validateSubnetsAreNotOverlapping(this.subnetAddresses);
+ }
+
+ @NonNull
+ @Override
+ public InetSocketAddress translate(@NonNull InetSocketAddress address) {
+ InetSocketAddress translatedAddress = null;
+ for (SubnetAddress subnetAddress : subnetAddresses) {
+ if (subnetAddress.contains(address)) {
+ translatedAddress = subnetAddress.address;
+ }
+ }
+ if (translatedAddress == null && defaultAddress.isPresent()) {
+ translatedAddress = defaultAddress.get();
+ }
+ if (translatedAddress == null) {
+ translatedAddress = address;
+ }
+ LOG.debug("[{}] Resolved {} to {}", logPrefix, address, translatedAddress);
+ return translatedAddress;
+ }
+
+ @Override
+ public void close() {}
+
+ private static InetSocketAddress parseAddress(String address) {
+ List<String> addressTuple = Splitter.onPattern(DELIMITER).splitToList(address);
+ if (addressTuple.size() == 2) {
+ return new InetSocketAddress(addressTuple.get(0), Integer.parseInt(addressTuple.get(1)));
Review Comment:
What is actually the downside of resolving on each #translate? I see 2 potential concerns:
1. Performance, which is negligible cause we don't do it that frequently (not on every query, only on metadata refreshes) and also we run it on a separate admin thread, which doesn't explicitly impact queries performance.
2. On metadata refreshes, if we resolve endpoint of existing Node to different IP, then we overwrite it in the Node object and the code that relies on search by EndPoint rather than by host ID (or do similar quirks) might get things wrong (maybe we don't have it, but it feels like such assumptions might have been somewhere, I'm just not yet that much familiar with the code). Do you remember anything of that sort in the codebase?
To conclude, if we go with either `resolve once on session init (translator constructor)` or `resolve on each connection`, we guard against both.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "jahstreet (via GitHub)" <gi...@apache.org>.
jahstreet commented on code in PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#discussion_r2009091709
##########
core/pom.xml:
##########
@@ -116,6 +116,11 @@
<groupId>org.reactivestreams</groupId>
<artifactId>reactive-streams</artifactId>
</dependency>
+ <dependency>
+ <groupId>com.github.seancfoley</groupId>
+ <artifactId>ipaddress</artifactId>
+ <optional>true</optional>
+ </dependency>
Review Comment:
A bit of vibe-coding and we can have it with around 100 lines of code. Will work on integrating a change.
##########
core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/SubnetAddressTranslator.java:
##########
@@ -0,0 +1,238 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.datastax.oss.driver.internal.core.addresstranslation;
+
+import com.datastax.oss.driver.api.core.addresstranslation.AddressTranslator;
+import com.datastax.oss.driver.api.core.config.DriverOption;
+import com.datastax.oss.driver.api.core.context.DriverContext;
+import com.datastax.oss.driver.internal.core.util.AddressUtils;
+import edu.umd.cs.findbugs.annotations.NonNull;
+import edu.umd.cs.findbugs.annotations.Nullable;
+import inet.ipaddr.IPAddress;
+import inet.ipaddr.IPAddressString;
+import java.net.InetSocketAddress;
+import java.util.List;
+import java.util.Optional;
+import java.util.stream.Collectors;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This translator returns the proxy address of the private subnet containing the Cassandra node IP,
+ * or default address if no matching subnets, or passes through the original node address if no
+ * default configured.
+ *
+ * <p>The translator can be used for scenarios when all nodes are behind some kind of proxy, and
+ * that proxy is different for nodes located in different subnets (eg. when Cassandra is deployed in
+ * multiple datacenters/regions). One can use this, for example, for Cassandra on Kubernetes with
+ * different Cassandra datacenters deployed to different Kubernetes clusters.
+ */
+public class SubnetAddressTranslator implements AddressTranslator {
+ private static final Logger LOG = LoggerFactory.getLogger(SubnetAddressTranslator.class);
+
+ /**
+ * A map of Cassandra node subnets (CIDR notations) to target addresses, for example (note quoted
+ * keys):
+ *
+ * <pre>
+ * advanced.address-translator.subnet-addresses {
+ * "100.64.0.0/15" = "cassandra.datacenter1.com:9042"
+ * "100.66.0.0/15" = "cassandra.datacenter2.com:9042"
+ * # IPv6 example:
+ * # "::ffff:6440:0/111" = "cassandra.datacenter1.com:9042"
+ * # "::ffff:6442:0/111" = "cassandra.datacenter2.com:9042"
+ * }
+ * </pre>
+ *
+ * Note: subnets must be represented as prefix blocks, see {@link
+ * inet.ipaddr.Address#isPrefixBlock()}.
+ */
+ public static final String ADDRESS_TRANSLATOR_SUBNET_ADDRESSES =
+ "advanced.address-translator.subnet-addresses";
+
+ /**
+ * A default address to fallback to if Cassandra node IP isn't contained in any of the configured
+ * subnets.
+ */
+ public static final String ADDRESS_TRANSLATOR_DEFAULT_ADDRESS =
+ "advanced.address-translator.default-address";
+
+ /**
+ * Whether to resolve the addresses on initialization (if true) or on each node (re-)connection
+ * (if false). Defaults to false.
+ */
+ public static final String ADDRESS_TRANSLATOR_RESOLVE_ADDRESSES =
+ "advanced.address-translator.resolve-addresses";
+
+ public static DriverOption ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_SUBNET_ADDRESSES;
+ }
+ };
+
+ public static DriverOption ADDRESS_TRANSLATOR_DEFAULT_ADDRESS_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_DEFAULT_ADDRESS;
+ }
+ };
+
+ public static DriverOption ADDRESS_TRANSLATOR_RESOLVE_ADDRESSES_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_RESOLVE_ADDRESSES;
+ }
+ };
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#discussion_r1956486693
##########
core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/SubnetAddressTranslator.java:
##########
@@ -0,0 +1,226 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.datastax.oss.driver.internal.core.addresstranslation;
+
+import com.datastax.oss.driver.api.core.addresstranslation.AddressTranslator;
+import com.datastax.oss.driver.api.core.config.DriverOption;
+import com.datastax.oss.driver.api.core.context.DriverContext;
+import com.google.common.base.Splitter;
+import edu.umd.cs.findbugs.annotations.NonNull;
+import inet.ipaddr.IPAddress;
+import inet.ipaddr.IPAddressString;
+import java.net.InetSocketAddress;
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.stream.Collectors;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This translator returns the proxy address of the private subnet containing the Cassandra node IP,
+ * or default address if no matching subnets, or passes through the original node address if no
+ * default configured.
+ *
+ * <p>The translator can be used for scenarios when all nodes are behind some kind of proxy, and
+ * that proxy is different for nodes located in different subnets (eg. when Cassandra is deployed in
+ * multiple datacenters/regions). One can use this, for example, for Cassandra on Kubernetes with
+ * different Cassandra datacenters deployed to different Kubernetes clusters.
+ */
+public class SubnetAddressTranslator implements AddressTranslator {
+
+ private static final Logger LOG = LoggerFactory.getLogger(SubnetAddressTranslator.class);
+
+ /**
+ * A map of Cassandra node subnets (CIDR notations) to target addresses, for example (note quoted
+ * keys):
+ *
+ * <pre>
+ * advanced.address-translator.subnet-addresses {
+ * "100.64.0.0/15" = "cassandra.datacenter1.com:9042"
+ * "100.66.0.0/15" = "cassandra.datacenter2.com"
+ * # IPv6 example:
+ * # "::ffff:6440:0/111" = "cassandra.datacenter1.com:9042"
+ * # "::ffff:6442:0/111" = "cassandra.datacenter2.com"
+ * }
+ * </pre>
+ *
+ * If configured without port, the default 9042 will be used. Also supports IPv6 subnets. Note:
+ * subnets must be represented as prefix blocks, see {@link inet.ipaddr.Address#isPrefixBlock()}.
+ */
+ public static final String ADDRESS_TRANSLATOR_SUBNET_ADDRESSES =
+ "advanced.address-translator.subnet-addresses";
+
+ /**
+ * A default address to fallback to if Cassandra node IP isn't contained in any of the configured
+ * subnets. If configured without port, the default 9042 will be used. Also supports IPv6
+ * addresses.
+ */
+ public static final String ADDRESS_TRANSLATOR_DEFAULT_ADDRESS =
+ "advanced.address-translator.default-address";
+
+ public static DriverOption ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_SUBNET_ADDRESSES;
+ }
+ };
+
+ public static DriverOption ADDRESS_TRANSLATOR_DEFAULT_ADDRESS_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_DEFAULT_ADDRESS;
+ }
+ };
+
+ private static final String DELIMITER = ":";
+ private static final int DEFAULT_PORT = 9042;
+
+ private final List<SubnetAddress> subnetAddresses;
+ private final Optional<InetSocketAddress> defaultAddress;
+ private final String logPrefix;
+
+ public SubnetAddressTranslator(@NonNull DriverContext context) {
+ logPrefix = context.getSessionName();
+ this.subnetAddresses =
+ context.getConfig().getDefaultProfile()
+ .getStringMap(ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION).entrySet().stream()
+ .map(
+ e -> {
+ // Quoted and/or containing forward slashes map keys in reference.conf are read to
+ // strings with additional quotes, eg. 100.64.0.0/15 -> '100.64.0."0/15"' or
+ // "100.64.0.0/15" -> '"100.64.0.0/15"'
+ String subnet = e.getKey().replaceAll("\"", "");
+ String address = e.getValue();
+ return new SubnetAddress(subnet, address);
+ })
+ .collect(Collectors.toList());
+ this.defaultAddress =
+ Optional.ofNullable(
+ context
+ .getConfig()
+ .getDefaultProfile()
+ .getString(ADDRESS_TRANSLATOR_DEFAULT_ADDRESS_OPTION, null))
+ .map(SubnetAddressTranslator::parseAddress);
+ SubnetAddressTranslator.validateSubnetsAreNotOverlapping(this.subnetAddresses);
+ }
+
+ @NonNull
+ @Override
+ public InetSocketAddress translate(@NonNull InetSocketAddress address) {
+ InetSocketAddress translatedAddress = null;
+ for (SubnetAddress subnetAddress : subnetAddresses) {
+ if (subnetAddress.contains(address)) {
+ translatedAddress = subnetAddress.address;
+ }
+ }
+ if (translatedAddress == null && defaultAddress.isPresent()) {
+ translatedAddress = defaultAddress.get();
+ }
+ if (translatedAddress == null) {
+ translatedAddress = address;
+ }
+ LOG.debug("[{}] Resolved {} to {}", logPrefix, address, translatedAddress);
+ return translatedAddress;
+ }
+
+ @Override
+ public void close() {}
+
+ private static InetSocketAddress parseAddress(String address) {
+ List<String> addressTuple = Splitter.onPattern(DELIMITER).splitToList(address);
+ if (addressTuple.size() == 2) {
+ return new InetSocketAddress(addressTuple.get(0), Integer.parseInt(addressTuple.get(1)));
Review Comment:
The main issue would just be load on DNS servers, this likely wouldn't be a problem in k8s as most implementations will have use a local cache; I remember the behavior with the 3.x driver where on any topology change in the cluster, it would provoke a refresh, which would cause a translate for each existing node which can cause a lot of load on a DNS server if you have X00 nodes and X000 clients. I think the same behavior exists with the 4.x.
In any case it's good to have a configurable knob to adjust the behavior if it could become a problem, and if the desire is to resolve from the start, to minimize how much you do it (since you really only need to do it once).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "absurdfarce (via GitHub)" <gi...@apache.org>.
absurdfarce commented on code in PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#discussion_r1999682162
##########
core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/SubnetAddressTranslator.java:
##########
@@ -0,0 +1,238 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.datastax.oss.driver.internal.core.addresstranslation;
+
+import com.datastax.oss.driver.api.core.addresstranslation.AddressTranslator;
+import com.datastax.oss.driver.api.core.config.DriverOption;
+import com.datastax.oss.driver.api.core.context.DriverContext;
+import com.datastax.oss.driver.internal.core.util.AddressUtils;
+import edu.umd.cs.findbugs.annotations.NonNull;
+import edu.umd.cs.findbugs.annotations.Nullable;
+import inet.ipaddr.IPAddress;
+import inet.ipaddr.IPAddressString;
+import java.net.InetSocketAddress;
+import java.util.List;
+import java.util.Optional;
+import java.util.stream.Collectors;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This translator returns the proxy address of the private subnet containing the Cassandra node IP,
+ * or default address if no matching subnets, or passes through the original node address if no
+ * default configured.
+ *
+ * <p>The translator can be used for scenarios when all nodes are behind some kind of proxy, and
+ * that proxy is different for nodes located in different subnets (eg. when Cassandra is deployed in
+ * multiple datacenters/regions). One can use this, for example, for Cassandra on Kubernetes with
+ * different Cassandra datacenters deployed to different Kubernetes clusters.
+ */
+public class SubnetAddressTranslator implements AddressTranslator {
+ private static final Logger LOG = LoggerFactory.getLogger(SubnetAddressTranslator.class);
+
+ /**
+ * A map of Cassandra node subnets (CIDR notations) to target addresses, for example (note quoted
+ * keys):
+ *
+ * <pre>
+ * advanced.address-translator.subnet-addresses {
+ * "100.64.0.0/15" = "cassandra.datacenter1.com:9042"
+ * "100.66.0.0/15" = "cassandra.datacenter2.com:9042"
+ * # IPv6 example:
+ * # "::ffff:6440:0/111" = "cassandra.datacenter1.com:9042"
+ * # "::ffff:6442:0/111" = "cassandra.datacenter2.com:9042"
+ * }
+ * </pre>
+ *
+ * Note: subnets must be represented as prefix blocks, see {@link
+ * inet.ipaddr.Address#isPrefixBlock()}.
+ */
+ public static final String ADDRESS_TRANSLATOR_SUBNET_ADDRESSES =
+ "advanced.address-translator.subnet-addresses";
+
+ /**
+ * A default address to fallback to if Cassandra node IP isn't contained in any of the configured
+ * subnets.
+ */
+ public static final String ADDRESS_TRANSLATOR_DEFAULT_ADDRESS =
+ "advanced.address-translator.default-address";
+
+ /**
+ * Whether to resolve the addresses on initialization (if true) or on each node (re-)connection
+ * (if false). Defaults to false.
+ */
+ public static final String ADDRESS_TRANSLATOR_RESOLVE_ADDRESSES =
+ "advanced.address-translator.resolve-addresses";
+
+ public static DriverOption ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_SUBNET_ADDRESSES;
+ }
+ };
+
+ public static DriverOption ADDRESS_TRANSLATOR_DEFAULT_ADDRESS_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_DEFAULT_ADDRESS;
+ }
+ };
+
+ public static DriverOption ADDRESS_TRANSLATOR_RESOLVE_ADDRESSES_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_RESOLVE_ADDRESSES;
+ }
+ };
Review Comment:
These DriverOptions need to be moved to DefaultDriverOption. We try to keep all the options for the basic driver there even if they're only enabled in certain cases. Note that the options for Astra connectivity [are also included there](https://github.com/apache/cassandra-java-driver/blob/4.19.0/core/src/main/java/com/datastax/oss/driver/api/core/config/DefaultDriverOption.java#L797-L803) even though not all users of the Java driver make use of Astra.
I'm guessing this usage was derived from Stefan's [example](https://github.com/apache/cassandra-java-driver/blob/7982f413a90935a91549aa3ee7cc64fa25d7c113/core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/FixedHostNameAddressTranslator.java#L43-L50) in FixedHostNameAddressTranslator. We need to clean that one up as well.
Also note that you'll want to add the new values to the DefaultDriverOption enum at the end. If you don't you'll get complaints from some of the build checks about potential disruptions to ordinal-based access of the enum. See [this commit](https://github.com/apache/cassandra-java-driver/commit/7982f413a90935a91549aa3ee7cc64fa25d7c113) for more context.
##########
core/pom.xml:
##########
@@ -116,6 +116,11 @@
<groupId>org.reactivestreams</groupId>
<artifactId>reactive-streams</artifactId>
</dependency>
+ <dependency>
+ <groupId>com.github.seancfoley</groupId>
+ <artifactId>ipaddress</artifactId>
+ <optional>true</optional>
+ </dependency>
Review Comment:
I really don't love the inclusion of _another_ dependency here, even if it's an optional one. It's only used in one class (near as I can tell)... is there really no way to get the functionality we need without adding this in?
##########
core/src/main/java/com/datastax/oss/driver/internal/core/ContactPoints.java:
##########
@@ -41,7 +38,22 @@ public static Set<EndPoint> merge(
Set<EndPoint> result = Sets.newHashSet(programmaticContactPoints);
for (String spec : configContactPoints) {
- for (InetSocketAddress address : extract(spec, resolve)) {
+
+ Set<InetSocketAddress> addresses = Collections.emptySet();
+ try {
+ addresses = AddressUtils.extract(spec, resolve);
+ } catch (RuntimeException e) {
+ LOG.warn("Ignoring invalid contact point {} ({})", spec, e.getMessage(), e);
+ }
Review Comment:
As I look at this now it feels like we had to make this a bit more complicated because AddressUtils.extract() now throws exceptions in most cases rather than just logging errors and returning an empty set. Was there a particular reason for this change? It's not immediately clear the exceptions buy you much here.
##########
core/src/main/java/com/datastax/oss/driver/internal/core/ContactPoints.java:
##########
@@ -41,7 +38,22 @@ public static Set<EndPoint> merge(
Set<EndPoint> result = Sets.newHashSet(programmaticContactPoints);
for (String spec : configContactPoints) {
- for (InetSocketAddress address : extract(spec, resolve)) {
+
+ Set<InetSocketAddress> addresses = Collections.emptySet();
+ try {
+ addresses = AddressUtils.extract(spec, resolve);
+ } catch (RuntimeException e) {
+ LOG.warn("Ignoring invalid contact point {} ({})", spec, e.getMessage(), e);
+ }
+
+ if (addresses.size() > 1) {
+ LOG.info(
+ "Contact point {} resolves to multiple addresses, will use them all ({})",
+ spec,
+ addresses);
+ }
Review Comment:
Does this log message offer us much useful information?
##########
core/src/main/java/com/datastax/oss/driver/internal/core/ContactPoints.java:
##########
@@ -41,7 +38,22 @@ public static Set<EndPoint> merge(
Set<EndPoint> result = Sets.newHashSet(programmaticContactPoints);
for (String spec : configContactPoints) {
- for (InetSocketAddress address : extract(spec, resolve)) {
+
+ Set<InetSocketAddress> addresses = Collections.emptySet();
+ try {
+ addresses = AddressUtils.extract(spec, resolve);
+ } catch (RuntimeException e) {
+ LOG.warn("Ignoring invalid contact point {} ({})", spec, e.getMessage(), e);
+ }
Review Comment:
Could just continue here to next iteration of the outer for loop. You know addresses is the empty set at this point so there's no point iterating over it below.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator to translate Cassandra node IPs from private network based on its subnet mask [cassandra-java-driver]
Posted by "absurdfarce (via GitHub)" <gi...@apache.org>.
absurdfarce commented on PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#issuecomment-2964043777
Hey @jahstreet, the Jenkins run is still ongoing but I'm seeing multiple test failures for SubnetTest and SubnetAddressTest. Haven't looked into it too deeply but most (all?) of the failures look to be from an empty string being passed to the Integer.parseInt() call in Subnet.parse(). I was able to repro immediately by running the test locally in my IDE.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "absurdfarce (via GitHub)" <gi...@apache.org>.
absurdfarce commented on code in PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#discussion_r2127402217
##########
core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/Subnet.java:
##########
@@ -0,0 +1,157 @@
+package com.datastax.oss.driver.internal.core.addresstranslation;
+
+import com.datastax.oss.driver.shaded.guava.common.annotations.VisibleForTesting;
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.util.Arrays;
+
+class Subnet {
+ private final byte[] subnet;
+ private final byte[] networkMask;
+ private final byte[] upper;
+ private final byte[] lower;
+
+ private Subnet(byte[] subnet, byte[] networkMask) {
+ this.subnet = subnet;
+ this.networkMask = networkMask;
+
+ byte[] upper = new byte[subnet.length];
+ byte[] lower = new byte[subnet.length];
+ for (int i = 0; i < subnet.length; i++) {
+ upper[i] = (byte) (subnet[i] | ~networkMask[i]);
+ lower[i] = (byte) (subnet[i] & networkMask[i]);
+ }
+ this.upper = upper;
+ this.lower = lower;
+ }
+
+ static Subnet parse(String subnetCIDR) throws UnknownHostException {
+ String[] parts = subnetCIDR.split("/");
Review Comment:
ErrorProne complains about this line when building due to [known issues with String.split()](https://errorprone.info/bugpattern/StringSplitter). Recommendation is to replace this with:
```java
List<String> parts = Splitter.on('/').splitToList(subnetCIDR);
```
I used something slightly different to preserve the `String[]` type throughout the rest of the code and it seemed to work fine:
```java
String[] parts = Iterables.toArray(Splitter.on('/').split(subnetCIDR), String.class);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "jahstreet (via GitHub)" <gi...@apache.org>.
jahstreet commented on code in PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#discussion_r1952404526
##########
core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/SubnetAddressTranslator.java:
##########
@@ -0,0 +1,210 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.datastax.oss.driver.internal.core.addresstranslation;
+
+import com.datastax.oss.driver.api.core.addresstranslation.AddressTranslator;
+import com.datastax.oss.driver.api.core.config.DriverOption;
+import com.datastax.oss.driver.api.core.context.DriverContext;
+import edu.umd.cs.findbugs.annotations.NonNull;
+import java.net.InetSocketAddress;
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.stream.Collectors;
+import org.apache.commons.net.util.SubnetUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This translator returns the proxy address of the private subnet containing the Cassandra node IP,
+ * or default address if no matching subnets, or passes through the original node address if no
+ * default configured.
+ *
+ * <p>The translator can be used for scenarios when all nodes are behind some kind of proxy, and
+ * that proxy is different for nodes located in different subnets (eg. when Cassandra is deployed in
+ * multiple datacenters/regions). One can use this, for example, for Cassandra on Kubernetes with
+ * different Cassandra datacenters deployed to different Kubernetes clusters.
+ */
+public class SubnetAddressTranslator implements AddressTranslator {
+
+ private static final Logger LOG = LoggerFactory.getLogger(SubnetAddressTranslator.class);
+
+ /**
+ * A comma separated list with Cassandra node subnets (CIDR notations) to target addresses in a
+ * format `<subnet>:<hostname>:<port>`, for example:
+ * `100.64.0.0/15:cassandra.datacenter1.com:9042,100.66.0.0/15:cassandra.datacenter1.com:9042`. If
+ * configured without port, the default 9042 will be used.
+ */
+ public static final String ADDRESS_TRANSLATOR_SUBNET_ADDRESSES =
+ "advanced.address-translator.subnet-addresses";
+ /**
+ * A default address to fallback to if Cassandra node IP isn't contained in any of the configured
+ * subnets. If configured without port, the default 9042 will be used.
+ */
+ public static final String ADDRESS_TRANSLATOR_DEFAULT_ADDRESS =
+ "advanced.address-translator.default-address";
+
+ public static DriverOption ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_SUBNET_ADDRESSES;
+ }
+ };
+ public static DriverOption ADDRESS_TRANSLATOR_DEFAULT_ADDRESS_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_DEFAULT_ADDRESS;
+ }
+ };
+
+ private static final String DELIMITER = ":";
+ private static final int DEFAULT_PORT = 9042;
+
+ private final List<SubnetAddress> subnetAddresses;
+ private final Optional<InetSocketAddress> defaultAddress;
+ private final String logPrefix;
+
+ public SubnetAddressTranslator(@NonNull DriverContext context) {
+ logPrefix = context.getSessionName();
+ this.subnetAddresses =
+ context.getConfig().getDefaultProfile()
+ .getStringList(ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION).stream()
Review Comment:
Okay, it is possible, just quoting works not intuitively, but for users it is invisible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "jahstreet (via GitHub)" <gi...@apache.org>.
jahstreet commented on code in PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#discussion_r1954008765
##########
core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/SubnetAddressTranslator.java:
##########
@@ -0,0 +1,226 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.datastax.oss.driver.internal.core.addresstranslation;
+
+import com.datastax.oss.driver.api.core.addresstranslation.AddressTranslator;
+import com.datastax.oss.driver.api.core.config.DriverOption;
+import com.datastax.oss.driver.api.core.context.DriverContext;
+import com.google.common.base.Splitter;
+import edu.umd.cs.findbugs.annotations.NonNull;
+import inet.ipaddr.IPAddress;
+import inet.ipaddr.IPAddressString;
+import java.net.InetSocketAddress;
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.stream.Collectors;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This translator returns the proxy address of the private subnet containing the Cassandra node IP,
+ * or default address if no matching subnets, or passes through the original node address if no
+ * default configured.
+ *
+ * <p>The translator can be used for scenarios when all nodes are behind some kind of proxy, and
+ * that proxy is different for nodes located in different subnets (eg. when Cassandra is deployed in
+ * multiple datacenters/regions). One can use this, for example, for Cassandra on Kubernetes with
+ * different Cassandra datacenters deployed to different Kubernetes clusters.
+ */
+public class SubnetAddressTranslator implements AddressTranslator {
+
+ private static final Logger LOG = LoggerFactory.getLogger(SubnetAddressTranslator.class);
+
+ /**
+ * A map of Cassandra node subnets (CIDR notations) to target addresses, for example (note quoted
+ * keys):
+ *
+ * <pre>
+ * advanced.address-translator.subnet-addresses {
+ * "100.64.0.0/15" = "cassandra.datacenter1.com:9042"
+ * "100.66.0.0/15" = "cassandra.datacenter2.com"
+ * # IPv6 example:
+ * # "::ffff:6440:0/111" = "cassandra.datacenter1.com:9042"
+ * # "::ffff:6442:0/111" = "cassandra.datacenter2.com"
+ * }
+ * </pre>
+ *
+ * If configured without port, the default 9042 will be used. Also supports IPv6 subnets. Note:
+ * subnets must be represented as prefix blocks, see {@link inet.ipaddr.Address#isPrefixBlock()}.
+ */
+ public static final String ADDRESS_TRANSLATOR_SUBNET_ADDRESSES =
+ "advanced.address-translator.subnet-addresses";
+
+ /**
+ * A default address to fallback to if Cassandra node IP isn't contained in any of the configured
+ * subnets. If configured without port, the default 9042 will be used. Also supports IPv6
+ * addresses.
+ */
+ public static final String ADDRESS_TRANSLATOR_DEFAULT_ADDRESS =
+ "advanced.address-translator.default-address";
+
+ public static DriverOption ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_SUBNET_ADDRESSES;
+ }
+ };
+
+ public static DriverOption ADDRESS_TRANSLATOR_DEFAULT_ADDRESS_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_DEFAULT_ADDRESS;
+ }
+ };
+
+ private static final String DELIMITER = ":";
+ private static final int DEFAULT_PORT = 9042;
+
+ private final List<SubnetAddress> subnetAddresses;
+ private final Optional<InetSocketAddress> defaultAddress;
+ private final String logPrefix;
+
+ public SubnetAddressTranslator(@NonNull DriverContext context) {
+ logPrefix = context.getSessionName();
+ this.subnetAddresses =
+ context.getConfig().getDefaultProfile()
+ .getStringMap(ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION).entrySet().stream()
+ .map(
+ e -> {
+ // Quoted and/or containing forward slashes map keys in reference.conf are read to
+ // strings with additional quotes, eg. 100.64.0.0/15 -> '100.64.0."0/15"' or
+ // "100.64.0.0/15" -> '"100.64.0.0/15"'
+ String subnet = e.getKey().replaceAll("\"", "");
+ String address = e.getValue();
+ return new SubnetAddress(subnet, address);
+ })
+ .collect(Collectors.toList());
+ this.defaultAddress =
+ Optional.ofNullable(
+ context
+ .getConfig()
+ .getDefaultProfile()
+ .getString(ADDRESS_TRANSLATOR_DEFAULT_ADDRESS_OPTION, null))
+ .map(SubnetAddressTranslator::parseAddress);
+ SubnetAddressTranslator.validateSubnetsAreNotOverlapping(this.subnetAddresses);
+ }
+
+ @NonNull
+ @Override
+ public InetSocketAddress translate(@NonNull InetSocketAddress address) {
+ InetSocketAddress translatedAddress = null;
+ for (SubnetAddress subnetAddress : subnetAddresses) {
+ if (subnetAddress.contains(address)) {
+ translatedAddress = subnetAddress.address;
+ }
+ }
+ if (translatedAddress == null && defaultAddress.isPresent()) {
+ translatedAddress = defaultAddress.get();
+ }
+ if (translatedAddress == null) {
+ translatedAddress = address;
+ }
+ LOG.debug("[{}] Resolved {} to {}", logPrefix, address, translatedAddress);
+ return translatedAddress;
+ }
+
+ @Override
+ public void close() {}
+
+ private static InetSocketAddress parseAddress(String address) {
+ List<String> addressTuple = Splitter.onPattern(DELIMITER).splitToList(address);
+ if (addressTuple.size() == 2) {
+ return new InetSocketAddress(addressTuple.get(0), Integer.parseInt(addressTuple.get(1)));
Review Comment:
I see your point. Indeed, depending on implementation it may happen so. Normally, a proxy (be it on K8s or not) is replicated for HA and replicas indeed can change IPs on restarts/migrations/maintenances or in other use-case-specific scenarios.
If we use `ContactPoints#extract`, then we can get multiple resolved IPs, while `InetSocketAddress#new` always resolves to a first one. I guess that is fine for us to also pick the first one from `ContactPoints#extract` 🤔 .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "absurdfarce (via GitHub)" <gi...@apache.org>.
absurdfarce commented on PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#issuecomment-2960783203
Bah, Jenkins failed with complaints like the following:
```
[2025-06-10T22:25:35.767Z] [INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ java-driver-core ---
[2025-06-10T22:25:35.767Z] [INFO] Compiling 796 source files to /home/jenkins/workspace/drivers_java_oss_PR-2013/core/target/classes
[2025-06-10T22:25:38.559Z] [INFO] -------------------------------------------------------------
[2025-06-10T22:25:38.559Z] [ERROR] COMPILATION ERROR :
[2025-06-10T22:25:38.559Z] [INFO] -------------------------------------------------------------
[2025-06-10T22:25:38.559Z] [ERROR] /home/jenkins/workspace/drivers_java_oss_PR-2013/core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/Subnet.java:[21,30] package com.google.common.base does not exist
[2025-06-10T22:25:38.559Z] [INFO] 1 error
[2025-06-10T22:25:38.559Z] [INFO] -------------------------------------------------------------
```
Weird thing was that local builds were just fine. This made absolutely no sense to me; I spent the afternoon trying various combinations of Maven versions and POM settings to see what might account for the difference.
Naturally the answer was pretty much right in front of me the whole time. This PR was branched off from a point in 4.x before [this commit](https://github.com/jahstreet/cassandra-java-driver/commit/d7e829775c4956d10c888c86b653f7ac2d10fa4b) went in. And that commit fixes precisely this behaviour. Once I included this change in my local checkout of the PR branch I was able to easily reproduce the failure above in my local build.
@jahstreet can you merge 4.x into your PR branch so that we can get this fix on your branch as well? I think such a merge + the other work you've already done should enable us to run a full build on our Jenkins server.
Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "jahstreet (via GitHub)" <gi...@apache.org>.
jahstreet commented on code in PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#discussion_r2000568199
##########
core/src/main/java/com/datastax/oss/driver/internal/core/ContactPoints.java:
##########
@@ -41,7 +38,22 @@ public static Set<EndPoint> merge(
Set<EndPoint> result = Sets.newHashSet(programmaticContactPoints);
for (String spec : configContactPoints) {
- for (InetSocketAddress address : extract(spec, resolve)) {
+
+ Set<InetSocketAddress> addresses = Collections.emptySet();
+ try {
+ addresses = AddressUtils.extract(spec, resolve);
+ } catch (RuntimeException e) {
+ LOG.warn("Ignoring invalid contact point {} ({})", spec, e.getMessage(), e);
+ }
Review Comment:
Previously, `#extract` code was used only in this class and we logged errors together with reasons of these errors. Now the info about reasons is moved to until method, which is called from multiple places. In this class, I aimed to keep logging (as well as other functionality) as close to the origin as seemed possible to avoid opinionated refactoring, so I needed a way to get reasons of errors from the utility `#extract` to log them together with the context logs.
Happy to agree on the way it should look like and change accordingly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "jahstreet (via GitHub)" <gi...@apache.org>.
jahstreet commented on code in PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#discussion_r2000572246
##########
core/src/main/java/com/datastax/oss/driver/internal/core/ContactPoints.java:
##########
@@ -41,7 +38,22 @@ public static Set<EndPoint> merge(
Set<EndPoint> result = Sets.newHashSet(programmaticContactPoints);
for (String spec : configContactPoints) {
- for (InetSocketAddress address : extract(spec, resolve)) {
+
+ Set<InetSocketAddress> addresses = Collections.emptySet();
+ try {
+ addresses = AddressUtils.extract(spec, resolve);
+ } catch (RuntimeException e) {
+ LOG.warn("Ignoring invalid contact point {} ({})", spec, e.getMessage(), e);
+ }
+
+ if (addresses.size() > 1) {
+ LOG.info(
+ "Contact point {} resolves to multiple addresses, will use them all ({})",
+ spec,
+ addresses);
+ }
Review Comment:
Same as above, it was there so I kept it as is.
As for me, this log is a good additional info when debugging failed to connect issues. Like, one could be surprised to see the client failed to connect logs where contact points do not match the configured ones.
What is your opinion on the need of it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#discussion_r1956486693
##########
core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/SubnetAddressTranslator.java:
##########
@@ -0,0 +1,226 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.datastax.oss.driver.internal.core.addresstranslation;
+
+import com.datastax.oss.driver.api.core.addresstranslation.AddressTranslator;
+import com.datastax.oss.driver.api.core.config.DriverOption;
+import com.datastax.oss.driver.api.core.context.DriverContext;
+import com.google.common.base.Splitter;
+import edu.umd.cs.findbugs.annotations.NonNull;
+import inet.ipaddr.IPAddress;
+import inet.ipaddr.IPAddressString;
+import java.net.InetSocketAddress;
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.stream.Collectors;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This translator returns the proxy address of the private subnet containing the Cassandra node IP,
+ * or default address if no matching subnets, or passes through the original node address if no
+ * default configured.
+ *
+ * <p>The translator can be used for scenarios when all nodes are behind some kind of proxy, and
+ * that proxy is different for nodes located in different subnets (eg. when Cassandra is deployed in
+ * multiple datacenters/regions). One can use this, for example, for Cassandra on Kubernetes with
+ * different Cassandra datacenters deployed to different Kubernetes clusters.
+ */
+public class SubnetAddressTranslator implements AddressTranslator {
+
+ private static final Logger LOG = LoggerFactory.getLogger(SubnetAddressTranslator.class);
+
+ /**
+ * A map of Cassandra node subnets (CIDR notations) to target addresses, for example (note quoted
+ * keys):
+ *
+ * <pre>
+ * advanced.address-translator.subnet-addresses {
+ * "100.64.0.0/15" = "cassandra.datacenter1.com:9042"
+ * "100.66.0.0/15" = "cassandra.datacenter2.com"
+ * # IPv6 example:
+ * # "::ffff:6440:0/111" = "cassandra.datacenter1.com:9042"
+ * # "::ffff:6442:0/111" = "cassandra.datacenter2.com"
+ * }
+ * </pre>
+ *
+ * If configured without port, the default 9042 will be used. Also supports IPv6 subnets. Note:
+ * subnets must be represented as prefix blocks, see {@link inet.ipaddr.Address#isPrefixBlock()}.
+ */
+ public static final String ADDRESS_TRANSLATOR_SUBNET_ADDRESSES =
+ "advanced.address-translator.subnet-addresses";
+
+ /**
+ * A default address to fallback to if Cassandra node IP isn't contained in any of the configured
+ * subnets. If configured without port, the default 9042 will be used. Also supports IPv6
+ * addresses.
+ */
+ public static final String ADDRESS_TRANSLATOR_DEFAULT_ADDRESS =
+ "advanced.address-translator.default-address";
+
+ public static DriverOption ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_SUBNET_ADDRESSES;
+ }
+ };
+
+ public static DriverOption ADDRESS_TRANSLATOR_DEFAULT_ADDRESS_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_DEFAULT_ADDRESS;
+ }
+ };
+
+ private static final String DELIMITER = ":";
+ private static final int DEFAULT_PORT = 9042;
+
+ private final List<SubnetAddress> subnetAddresses;
+ private final Optional<InetSocketAddress> defaultAddress;
+ private final String logPrefix;
+
+ public SubnetAddressTranslator(@NonNull DriverContext context) {
+ logPrefix = context.getSessionName();
+ this.subnetAddresses =
+ context.getConfig().getDefaultProfile()
+ .getStringMap(ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION).entrySet().stream()
+ .map(
+ e -> {
+ // Quoted and/or containing forward slashes map keys in reference.conf are read to
+ // strings with additional quotes, eg. 100.64.0.0/15 -> '100.64.0."0/15"' or
+ // "100.64.0.0/15" -> '"100.64.0.0/15"'
+ String subnet = e.getKey().replaceAll("\"", "");
+ String address = e.getValue();
+ return new SubnetAddress(subnet, address);
+ })
+ .collect(Collectors.toList());
+ this.defaultAddress =
+ Optional.ofNullable(
+ context
+ .getConfig()
+ .getDefaultProfile()
+ .getString(ADDRESS_TRANSLATOR_DEFAULT_ADDRESS_OPTION, null))
+ .map(SubnetAddressTranslator::parseAddress);
+ SubnetAddressTranslator.validateSubnetsAreNotOverlapping(this.subnetAddresses);
+ }
+
+ @NonNull
+ @Override
+ public InetSocketAddress translate(@NonNull InetSocketAddress address) {
+ InetSocketAddress translatedAddress = null;
+ for (SubnetAddress subnetAddress : subnetAddresses) {
+ if (subnetAddress.contains(address)) {
+ translatedAddress = subnetAddress.address;
+ }
+ }
+ if (translatedAddress == null && defaultAddress.isPresent()) {
+ translatedAddress = defaultAddress.get();
+ }
+ if (translatedAddress == null) {
+ translatedAddress = address;
+ }
+ LOG.debug("[{}] Resolved {} to {}", logPrefix, address, translatedAddress);
+ return translatedAddress;
+ }
+
+ @Override
+ public void close() {}
+
+ private static InetSocketAddress parseAddress(String address) {
+ List<String> addressTuple = Splitter.onPattern(DELIMITER).splitToList(address);
+ if (addressTuple.size() == 2) {
+ return new InetSocketAddress(addressTuple.get(0), Integer.parseInt(addressTuple.get(1)));
Review Comment:
The main issue would just be load on DNS servers, this likely wouldn't be a problem in k8s as most implementations will have use a local cache; I remember the behavior with the 3.x driver where on any topology change in the cluster, it would provoke a refresh, which would cause a translate for each existing node which can cause a lot of load on a DNS server if you have X00 and X000 clients. I think the same behavior exists with the 4.x.
In any case it's good to have a configurable knob to adjust the behavior if it could become a problem, and if the desire is to resolve from the start, to minimize how much you do it (since you really only need to do it once).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "jahstreet (via GitHub)" <gi...@apache.org>.
jahstreet commented on code in PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#discussion_r1950473215
##########
core/pom.xml:
##########
@@ -116,6 +116,10 @@
<groupId>org.reactivestreams</groupId>
<artifactId>reactive-streams</artifactId>
</dependency>
+ <dependency>
+ <groupId>commons-net</groupId>
+ <artifactId>commons-net</artifactId>
+ </dependency>
Review Comment:
Changed to IPAddress and made it optional.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "absurdfarce (via GitHub)" <gi...@apache.org>.
absurdfarce commented on PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#issuecomment-2770830312
Apologies, this is on my list but I haven't made it back to reconsider the updated comments in this review. I appreciate your patience @jahstreet!
FWIW I have added this to the 4.19.1 release planning doc under the working assumption that we'll almost certainly get this in in some form we can all agree on.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator to translate Cassandra node IPs from private network based on its subnet mask [cassandra-java-driver]
Posted by "absurdfarce (via GitHub)" <gi...@apache.org>.
absurdfarce commented on PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#issuecomment-2964051660
Oh, I see... fortunately this one's an easy fix:
```diff
diff --git a/core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/Subnet.java b/core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/Subnet.java
index ec83626c5..7c25e94e2 100644
--- a/core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/Subnet.java
+++ b/core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/Subnet.java
@@ -45,7 +45,7 @@ class Subnet {
}
static Subnet parse(String subnetCIDR) throws UnknownHostException {
- List<String> parts = Splitter.on("/").splitToList("/");
+ List<String> parts = Splitter.on("/").splitToList(subnetCIDR);
if (parts.size() != 2) {
throw new IllegalArgumentException("Invalid subnet: " + subnetCIDR);
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "absurdfarce (via GitHub)" <gi...@apache.org>.
absurdfarce commented on PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#issuecomment-2960787743
Oh, almost forgot... the reason the build now fails with that change in place is because you're using the normal Guava packages [here](https://github.com/jahstreet/cassandra-java-driver/blob/add-subnet-address-translator/core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/Subnet.java#L21) (and potentially other places in your PR... I admit I haven't checked yet). This should be changed to use the shaded packages for Guava along the lines of the [VisibleForTesting import](https://github.com/jahstreet/cassandra-java-driver/blob/add-subnet-address-translator/core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/Subnet.java#L20) just above your addition.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator to translate Cassandra node IPs from private network based on its subnet mask [cassandra-java-driver]
Posted by "jahstreet (via GitHub)" <gi...@apache.org>.
jahstreet commented on PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#issuecomment-2965353683
There is some difference between my local env and CI, specifically in the way the DNS is resolved:
```
Expecting message to be:
"Configured subnets are overlapping: SubnetAddress[subnet=[100, 64, 0, 0], address=cassandra.datacenter1.com:19042], SubnetAddress[subnet=[100, 65, 0, 0], address=cassandra.datacenter2.com:19042]"
but was:
"Configured subnets are overlapping: SubnetAddress[subnet=[100, 64, 0, 0], address=cassandra.datacenter1.com/<unresolved>:19042], SubnetAddress[subnet=[100, 65, 0, 0], address=cassandra.datacenter2.com/<unresolved>:19042]"
```
I will give a thought today on how to approach it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "absurdfarce (via GitHub)" <gi...@apache.org>.
absurdfarce commented on code in PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#discussion_r2127542294
##########
core/src/main/java/com/datastax/oss/driver/api/core/config/DefaultDriverOption.java:
##########
@@ -994,7 +994,48 @@ public enum DefaultDriverOption implements DriverOption {
*
* <p>Value-type: boolean
*/
- SSL_ALLOW_DNS_REVERSE_LOOKUP_SAN("advanced.ssl-engine-factory.allow-dns-reverse-lookup-san");
+ SSL_ALLOW_DNS_REVERSE_LOOKUP_SAN("advanced.ssl-engine-factory.allow-dns-reverse-lookup-san"),
+ /**
+ * An address to always translate all node addresses to that same proxy hostname no matter what IP
+ * address a node has, but still using its native transport port.
+ *
+ * <p>Value-Type: {@link String}
+ */
+ ADDRESS_TRANSLATOR_ADVERTISED_HOSTNAME("advanced.address-translator.advertised-hostname"),
+ /**
+ * A map of Cassandra node subnets (CIDR notations) to target addresses, for example (note quoted
+ * keys):
+ *
+ * <pre>
+ * advanced.address-translator.subnet-addresses {
+ * "100.64.0.0/15" = "cassandra.datacenter1.com:9042"
+ * "100.66.0.0/15" = "cassandra.datacenter2.com:9042"
+ * # IPv6 example:
+ * # "::ffff:6440:0/111" = "cassandra.datacenter1.com:9042"
+ * # "::ffff:6442:0/111" = "cassandra.datacenter2.com:9042"
+ * }
+ * </pre>
+ *
+ * Note: subnets must be represented as prefix blocks, see {@link
+ * inet.ipaddr.Address#isPrefixBlock()}.
+ *
+ * <p>Value type: {@link java.util.Map Map}<{@link String},{@link String}>
+ */
+ ADDRESS_TRANSLATOR_SUBNET_ADDRESSES("advanced.address-translator.subnet-addresses"),
+ /**
+ * A default address to fallback to if Cassandra node IP isn't contained in any of the configured
+ * subnets.
+ *
+ * <p>Value-Type: {@link String}
+ */
+ ADDRESS_TRANSLATOR_DEFAULT_ADDRESS("advanced.address-translator.default-address"),
+ /**
+ * Whether to resolve the addresses on initialization (if true) or on each node (re-)connection
+ * (if false). Defaults to false.
+ *
+ * <p>Value-Type: boolean
+ */
+ ADDRESS_TRANSLATOR_RESOLVE_ADDRESSES("advanced.address-translator.resolve-addresses");
Review Comment:
TypedDriverOptionTest [will fail](https://github.com/apache/cassandra-java-driver/blob/4.19.0/core/src/test/java/com/datastax/oss/driver/api/core/config/TypedDriverOptionTest.java#L35) if this enum contains values which aren't also in TypedDriverOption. The following diff should get you past this issue:
```diff
diff --git a/core/src/main/java/com/datastax/oss/driver/api/core/config/TypedDriverOption.java b/core/src/main/java/com/datastax/oss/driver/api/core/config/TypedDriverOption.java
index 93e2b4684..aa4e4af12 100644
--- a/core/src/main/java/com/datastax/oss/driver/api/core/config/TypedDriverOption.java
+++ b/core/src/main/java/com/datastax/oss/driver/api/core/config/TypedDriverOption.java
@@ -896,6 +896,20 @@ public class TypedDriverOption<ValueT> {
DefaultDriverOption.LOAD_BALANCING_DC_FAILOVER_ALLOW_FOR_LOCAL_CONSISTENCY_LEVELS,
GenericType.BOOLEAN);
+ public static final TypedDriverOption<String> ADDRESS_TRANSLATOR_ADVERTISED_HOSTNAME =
+ new TypedDriverOption<>(
+ DefaultDriverOption.ADDRESS_TRANSLATOR_ADVERTISED_HOSTNAME, GenericType.STRING);
+ public static final TypedDriverOption<Map<String, String>> ADDRESS_TRANSLATOR_SUBNET_ADDRESSES =
+ new TypedDriverOption<>(
+ DefaultDriverOption.ADDRESS_TRANSLATOR_SUBNET_ADDRESSES,
+ GenericType.mapOf(GenericType.STRING, GenericType.STRING));
+ public static final TypedDriverOption<String> ADDRESS_TRANSLATOR_DEFAULT_ADDRESS =
+ new TypedDriverOption<>(
+ DefaultDriverOption.ADDRESS_TRANSLATOR_DEFAULT_ADDRESS, GenericType.STRING);
+ public static final TypedDriverOption<Boolean> ADDRESS_TRANSLATOR_RESOLVE_ADDRESSES =
+ new TypedDriverOption<>(
+ DefaultDriverOption.ADDRESS_TRANSLATOR_RESOLVE_ADDRESSES, GenericType.BOOLEAN);
+
/**
* Ordered preference list of remote dcs optionally supplied for automatic failover and included
* in query plan. This feature is enabled only when max-nodes-per-remote-dc is greater than 0.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "absurdfarce (via GitHub)" <gi...@apache.org>.
absurdfarce commented on code in PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#discussion_r2127385733
##########
core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/Subnet.java:
##########
@@ -0,0 +1,157 @@
+package com.datastax.oss.driver.internal.core.addresstranslation;
Review Comment:
Need ASF license text at the top of this source file
##########
core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/SubnetAddress.java:
##########
@@ -0,0 +1,48 @@
+package com.datastax.oss.driver.internal.core.addresstranslation;
Review Comment:
Need ASF license text at the top of this source file here as well
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator to translate Cassandra node IPs from private network based on its subnet mask [cassandra-java-driver]
Posted by "jahstreet (via GitHub)" <gi...@apache.org>.
jahstreet commented on code in PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#discussion_r2139431307
##########
core/src/test/java/com/datastax/oss/driver/internal/core/addresstranslation/SubnetAddressTranslatorTest.java:
##########
@@ -0,0 +1,145 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.datastax.oss.driver.internal.core.addresstranslation;
+
+import static com.datastax.oss.driver.api.core.config.DefaultDriverOption.ADDRESS_TRANSLATOR_DEFAULT_ADDRESS;
+import static com.datastax.oss.driver.api.core.config.DefaultDriverOption.ADDRESS_TRANSLATOR_SUBNET_ADDRESSES;
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatIllegalArgumentException;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
+
+import com.datastax.oss.driver.api.core.config.DriverExecutionProfile;
+import com.datastax.oss.driver.internal.core.context.DefaultDriverContext;
+import com.datastax.oss.driver.internal.core.context.MockedDriverContextFactory;
+import com.google.common.collect.ImmutableMap;
Review Comment:
```suggestion
com.datastax.oss.driver.shaded.guava.common.collect.ImmutableMap;
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "jahstreet (via GitHub)" <gi...@apache.org>.
jahstreet commented on code in PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#discussion_r2009087227
##########
core/pom.xml:
##########
@@ -116,6 +116,11 @@
<groupId>org.reactivestreams</groupId>
<artifactId>reactive-streams</artifactId>
</dependency>
+ <dependency>
+ <groupId>com.github.seancfoley</groupId>
+ <artifactId>ipaddress</artifactId>
+ <optional>true</optional>
+ </dependency>
Review Comment:
We need at least the following functionality to work with subnets here:
- Validate subnet string is in a prefix block format
- Check if subnet contains IP address
- All for IPv4 and IPv6
The library is quite big, so copying over its parts is an overkill.
Then the alternative is to implement these functions ourselves.
Looking into it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "jahstreet (via GitHub)" <gi...@apache.org>.
jahstreet commented on code in PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#discussion_r2000552728
##########
core/pom.xml:
##########
@@ -116,6 +116,11 @@
<groupId>org.reactivestreams</groupId>
<artifactId>reactive-streams</artifactId>
</dependency>
+ <dependency>
+ <groupId>com.github.seancfoley</groupId>
+ <artifactId>ipaddress</artifactId>
+ <optional>true</optional>
+ </dependency>
Review Comment:
First thing that comes to mind is implementing it ourselves (or in other words copying it over from the library). Lemme evaluate how much of the util code is needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "jahstreet (via GitHub)" <gi...@apache.org>.
jahstreet commented on code in PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#discussion_r1953990634
##########
manual/core/address_resolution/README.md:
##########
@@ -118,6 +118,52 @@ datastax-java-driver.advanced.address-translator.class = com.mycompany.MyAddress
Note: the contact points provided while creating the `CqlSession` are not translated, only addresses
retrieved from or sent by Cassandra nodes are.
+### Fixed proxy hostname
+
+If your client applications access Cassandra through some kind of proxy (eg. with AWS PrivateLink when all Cassandra
+nodes are exposed via one hostname pointing to AWS Endpoint), you can configure driver with
+`FixedHostNameAddressTranslator` to always translate all node addresses to that same proxy hostname, no matter what IP
+address a node has but still using its native transport port.
+
+To use it, specify the following in the [configuration](../configuration):
+
+```
+datastax-java-driver.advanced.address-translator.class = FixedHostNameAddressTranslator
+advertised-hostname = proxyhostname
+```
+
+### Fixed proxy hostname per subnet
+
+When running Cassandra in a private network and accessing it from outside of that private network via some kind of
+proxy, we have an option to use `FixedHostNameAddressTranslator`. But for multi-datacenter Cassandra deployments, we
+want to have more control over routing queries to a specific datacenter (eg. for optimizing latencies), which requires
+setting up a separate proxy per datacenter.
+
+Normally, each Cassandra datacenter nodes are deployed to a separate subnet to support internode communications in the
+cluster and avoid IP addresses collisions. So when Cassandra broadcasts its nodes IP addresses, we can determine which
+datacenter that node belongs to by checking its IP address against the given datacenter subnet.
+
+For such scenarios you can use `SubnetAddressTranslator` to translate node IPs to the datacenter proxy address
+associated with it.
+
+To use it, specify the following in the [configuration](../configuration):
+```
+datastax-java-driver.advanced.address-translator {
+ class = SubnetAddressTranslator
+ subnet-addresses {
+ "100.64.0.0/15" = "cassandra.datacenter1.com:9042"
+ "100.66.0.0/15" = "cassandra.datacenter2.com" # port defaults to 9042 if not specified
+ # IPv6 example:
+ # "::ffff:6440:0/111" = "cassandra.datacenter1.com:9042"
+ # "::ffff:6442:0/111" = "cassandra.datacenter2.com" # port defaults to 9042 if not specified
+ }
+ # Optional. When configured, addresses not matching the configured subnets are translated to it. Port defaults to 9042 if not specified.
+ default-address = "cassandra.datacenter1.com:9042"
+}
+```
+
+Such setup is common for running Cassandra on Kubernetes with [k8ssandra](https://docs.k8ssandra.io/).
Review Comment:
Right!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator to translate Cassandra node IPs from private network based on its subnet mask [cassandra-java-driver]
Posted by "absurdfarce (via GitHub)" <gi...@apache.org>.
absurdfarce commented on PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#issuecomment-2967968848
Okay, with your latest round of fixes I do get a good run against DataStax CI @jahstreet. There are indeed some test failures but they're all known problematic cases, mostly [CASSJAVA-95](https://issues.apache.org/jira/browse/CASSJAVA-95). Given these results I'm comfortable changing my review to an approval.
@tolbertam and/or @aratno ... I know one or both of you were planning on taking another look at this PR once we got it to a stable state. I think we're there now so if I can get another :+1: from one (or both) of you I think we're good to go here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "jahstreet (via GitHub)" <gi...@apache.org>.
jahstreet commented on code in PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#discussion_r1950482875
##########
core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/SubnetAddressTranslator.java:
##########
@@ -0,0 +1,210 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.datastax.oss.driver.internal.core.addresstranslation;
+
+import com.datastax.oss.driver.api.core.addresstranslation.AddressTranslator;
+import com.datastax.oss.driver.api.core.config.DriverOption;
+import com.datastax.oss.driver.api.core.context.DriverContext;
+import edu.umd.cs.findbugs.annotations.NonNull;
+import java.net.InetSocketAddress;
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.stream.Collectors;
+import org.apache.commons.net.util.SubnetUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This translator returns the proxy address of the private subnet containing the Cassandra node IP,
+ * or default address if no matching subnets, or passes through the original node address if no
+ * default configured.
+ *
+ * <p>The translator can be used for scenarios when all nodes are behind some kind of proxy, and
+ * that proxy is different for nodes located in different subnets (eg. when Cassandra is deployed in
+ * multiple datacenters/regions). One can use this, for example, for Cassandra on Kubernetes with
+ * different Cassandra datacenters deployed to different Kubernetes clusters.
+ */
+public class SubnetAddressTranslator implements AddressTranslator {
+
+ private static final Logger LOG = LoggerFactory.getLogger(SubnetAddressTranslator.class);
+
+ /**
+ * A comma separated list with Cassandra node subnets (CIDR notations) to target addresses in a
+ * format `<subnet>:<hostname>:<port>`, for example:
+ * `100.64.0.0/15:cassandra.datacenter1.com:9042,100.66.0.0/15:cassandra.datacenter1.com:9042`. If
+ * configured without port, the default 9042 will be used.
+ */
+ public static final String ADDRESS_TRANSLATOR_SUBNET_ADDRESSES =
+ "advanced.address-translator.subnet-addresses";
+ /**
+ * A default address to fallback to if Cassandra node IP isn't contained in any of the configured
+ * subnets. If configured without port, the default 9042 will be used.
+ */
+ public static final String ADDRESS_TRANSLATOR_DEFAULT_ADDRESS =
+ "advanced.address-translator.default-address";
+
+ public static DriverOption ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_SUBNET_ADDRESSES;
+ }
+ };
+ public static DriverOption ADDRESS_TRANSLATOR_DEFAULT_ADDRESS_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_DEFAULT_ADDRESS;
+ }
+ };
+
+ private static final String DELIMITER = ":";
+ private static final int DEFAULT_PORT = 9042;
+
+ private final List<SubnetAddress> subnetAddresses;
+ private final Optional<InetSocketAddress> defaultAddress;
+ private final String logPrefix;
+
+ public SubnetAddressTranslator(@NonNull DriverContext context) {
+ logPrefix = context.getSessionName();
+ this.subnetAddresses =
+ context.getConfig().getDefaultProfile()
+ .getStringList(ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION).stream()
+ .map(SubnetAddress::fromString)
+ .collect(Collectors.toList());
+ this.defaultAddress =
+ Optional.ofNullable(
+ context
+ .getConfig()
+ .getDefaultProfile()
+ .getString(ADDRESS_TRANSLATOR_DEFAULT_ADDRESS_OPTION, null))
+ .map(SubnetAddressTranslator::parseAddress);
+ SubnetAddressTranslator.validateSubnetsAreNotOverlapping(this.subnetAddresses);
+ }
+
+ @NonNull
+ @Override
+ public InetSocketAddress translate(@NonNull InetSocketAddress address) {
+ InetSocketAddress translatedAddress = null;
+ for (SubnetAddress subnetAddress : subnetAddresses) {
+ if (subnetAddress.contains(address)) {
+ translatedAddress = subnetAddress.address;
+ }
+ }
+ if (translatedAddress == null && defaultAddress.isPresent()) {
+ translatedAddress = defaultAddress.get();
+ }
+ if (translatedAddress == null) {
+ translatedAddress = address;
+ }
+ LOG.debug("[{}] Resolved {} to {}", logPrefix, address, translatedAddress);
+ return translatedAddress;
+ }
+
+ @Override
+ public void close() {}
+
+ private static InetSocketAddress parseAddress(String address) {
+ String[] addressTuple = address.split(DELIMITER);
Review Comment:
Addressed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#discussion_r1953319078
##########
core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/SubnetAddressTranslator.java:
##########
@@ -0,0 +1,226 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.datastax.oss.driver.internal.core.addresstranslation;
+
+import com.datastax.oss.driver.api.core.addresstranslation.AddressTranslator;
+import com.datastax.oss.driver.api.core.config.DriverOption;
+import com.datastax.oss.driver.api.core.context.DriverContext;
+import com.google.common.base.Splitter;
+import edu.umd.cs.findbugs.annotations.NonNull;
+import inet.ipaddr.IPAddress;
+import inet.ipaddr.IPAddressString;
+import java.net.InetSocketAddress;
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.stream.Collectors;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This translator returns the proxy address of the private subnet containing the Cassandra node IP,
+ * or default address if no matching subnets, or passes through the original node address if no
+ * default configured.
+ *
+ * <p>The translator can be used for scenarios when all nodes are behind some kind of proxy, and
+ * that proxy is different for nodes located in different subnets (eg. when Cassandra is deployed in
+ * multiple datacenters/regions). One can use this, for example, for Cassandra on Kubernetes with
+ * different Cassandra datacenters deployed to different Kubernetes clusters.
+ */
+public class SubnetAddressTranslator implements AddressTranslator {
+
+ private static final Logger LOG = LoggerFactory.getLogger(SubnetAddressTranslator.class);
+
+ /**
+ * A map of Cassandra node subnets (CIDR notations) to target addresses, for example (note quoted
+ * keys):
+ *
+ * <pre>
+ * advanced.address-translator.subnet-addresses {
+ * "100.64.0.0/15" = "cassandra.datacenter1.com:9042"
+ * "100.66.0.0/15" = "cassandra.datacenter2.com"
+ * # IPv6 example:
+ * # "::ffff:6440:0/111" = "cassandra.datacenter1.com:9042"
Review Comment:
:heart: appreciate the IPv6 support!
##########
core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/SubnetAddressTranslator.java:
##########
@@ -0,0 +1,226 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.datastax.oss.driver.internal.core.addresstranslation;
+
+import com.datastax.oss.driver.api.core.addresstranslation.AddressTranslator;
+import com.datastax.oss.driver.api.core.config.DriverOption;
+import com.datastax.oss.driver.api.core.context.DriverContext;
+import com.google.common.base.Splitter;
+import edu.umd.cs.findbugs.annotations.NonNull;
+import inet.ipaddr.IPAddress;
+import inet.ipaddr.IPAddressString;
+import java.net.InetSocketAddress;
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.stream.Collectors;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This translator returns the proxy address of the private subnet containing the Cassandra node IP,
+ * or default address if no matching subnets, or passes through the original node address if no
+ * default configured.
+ *
+ * <p>The translator can be used for scenarios when all nodes are behind some kind of proxy, and
+ * that proxy is different for nodes located in different subnets (eg. when Cassandra is deployed in
+ * multiple datacenters/regions). One can use this, for example, for Cassandra on Kubernetes with
+ * different Cassandra datacenters deployed to different Kubernetes clusters.
+ */
+public class SubnetAddressTranslator implements AddressTranslator {
+
+ private static final Logger LOG = LoggerFactory.getLogger(SubnetAddressTranslator.class);
+
+ /**
+ * A map of Cassandra node subnets (CIDR notations) to target addresses, for example (note quoted
+ * keys):
+ *
+ * <pre>
+ * advanced.address-translator.subnet-addresses {
+ * "100.64.0.0/15" = "cassandra.datacenter1.com:9042"
+ * "100.66.0.0/15" = "cassandra.datacenter2.com"
+ * # IPv6 example:
+ * # "::ffff:6440:0/111" = "cassandra.datacenter1.com:9042"
+ * # "::ffff:6442:0/111" = "cassandra.datacenter2.com"
+ * }
+ * </pre>
+ *
+ * If configured without port, the default 9042 will be used. Also supports IPv6 subnets. Note:
+ * subnets must be represented as prefix blocks, see {@link inet.ipaddr.Address#isPrefixBlock()}.
+ */
+ public static final String ADDRESS_TRANSLATOR_SUBNET_ADDRESSES =
+ "advanced.address-translator.subnet-addresses";
+
+ /**
+ * A default address to fallback to if Cassandra node IP isn't contained in any of the configured
+ * subnets. If configured without port, the default 9042 will be used. Also supports IPv6
Review Comment:
I was just looking over the existing configuration in reference.conf and it looks like we are pretty consistent about requiring the native port when it comes to specifying endpoints (e.g. `basic.contact-points` mandates this through `ContactPoints.extract`).
I think it may just be better to always require the port to be consistent with `basic.contact-points`, and it would also simplify the code a little bit.
##########
core/src/main/resources/reference.conf:
##########
@@ -1020,17 +1020,30 @@ datastax-java-driver {
# the package com.datastax.oss.driver.internal.core.addresstranslation.
#
# The driver provides the following implementations out of the box:
- # - PassThroughAddressTranslator: returns all addresses unchanged
+ # - PassThroughAddressTranslator: returns all addresses unchanged.
# - FixedHostNameAddressTranslator: translates all addresses to a specific hostname.
+ # - SubnetAddressTranslator: translates addresses to hostname based on the subnet matches.
# - Ec2MultiRegionAddressTranslator: suitable for an Amazon multi-region EC2 deployment where
# clients are also deployed in EC2. It optimizes network costs by favoring private IPs over
# public ones whenever possible.
#
# You can also specify a custom class that implements AddressTranslator and has a public
# constructor with a DriverContext argument.
class = PassThroughAddressTranslator
+ #
# This property has to be set only in case you use FixedHostNameAddressTranslator.
# advertised-hostname = mycustomhostname
+ #
+ # Theses properties have to be set only in case you use SubnetAddressTranslator.
Review Comment:
```suggestion
# These properties are only applicable in case you use SubnetAddressTranslator.
```
##########
core/src/main/resources/reference.conf:
##########
@@ -1020,17 +1020,30 @@ datastax-java-driver {
# the package com.datastax.oss.driver.internal.core.addresstranslation.
#
# The driver provides the following implementations out of the box:
- # - PassThroughAddressTranslator: returns all addresses unchanged
+ # - PassThroughAddressTranslator: returns all addresses unchanged.
# - FixedHostNameAddressTranslator: translates all addresses to a specific hostname.
+ # - SubnetAddressTranslator: translates addresses to hostname based on the subnet matches.
# - Ec2MultiRegionAddressTranslator: suitable for an Amazon multi-region EC2 deployment where
# clients are also deployed in EC2. It optimizes network costs by favoring private IPs over
# public ones whenever possible.
#
# You can also specify a custom class that implements AddressTranslator and has a public
# constructor with a DriverContext argument.
class = PassThroughAddressTranslator
+ #
# This property has to be set only in case you use FixedHostNameAddressTranslator.
# advertised-hostname = mycustomhostname
+ #
+ # Theses properties have to be set only in case you use SubnetAddressTranslator.
+ # subnet-addresses {
+ # "100.64.0.0/15" = "cassandra.datacenter1.com:9042"
+ # "100.66.0.0/15" = "cassandra.datacenter2.com" # port defaults to 9042 if not specified
+ # # IPv6 example:
+ # # "::ffff:6440:0/111" = "cassandra.datacenter1.com:9042"
+ # # "::ffff:6442:0/111" = "cassandra.datacenter2.com" # port defaults to 9042 if not specified
+ # }
+ # Optional. When configured, addresses not matching the configured subnets are translated to it. Port defaults to 9042 if not specified.
Review Comment:
```suggestion
# Optional. When configured, addresses not matching the configured subnets are translated to this address. Port defaults to 9042 if not specified.
```
##########
core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/SubnetAddressTranslator.java:
##########
@@ -0,0 +1,226 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.datastax.oss.driver.internal.core.addresstranslation;
+
+import com.datastax.oss.driver.api.core.addresstranslation.AddressTranslator;
+import com.datastax.oss.driver.api.core.config.DriverOption;
+import com.datastax.oss.driver.api.core.context.DriverContext;
+import com.google.common.base.Splitter;
+import edu.umd.cs.findbugs.annotations.NonNull;
+import inet.ipaddr.IPAddress;
+import inet.ipaddr.IPAddressString;
+import java.net.InetSocketAddress;
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.stream.Collectors;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This translator returns the proxy address of the private subnet containing the Cassandra node IP,
+ * or default address if no matching subnets, or passes through the original node address if no
+ * default configured.
+ *
+ * <p>The translator can be used for scenarios when all nodes are behind some kind of proxy, and
+ * that proxy is different for nodes located in different subnets (eg. when Cassandra is deployed in
+ * multiple datacenters/regions). One can use this, for example, for Cassandra on Kubernetes with
+ * different Cassandra datacenters deployed to different Kubernetes clusters.
+ */
+public class SubnetAddressTranslator implements AddressTranslator {
+
+ private static final Logger LOG = LoggerFactory.getLogger(SubnetAddressTranslator.class);
+
+ /**
+ * A map of Cassandra node subnets (CIDR notations) to target addresses, for example (note quoted
+ * keys):
+ *
+ * <pre>
+ * advanced.address-translator.subnet-addresses {
+ * "100.64.0.0/15" = "cassandra.datacenter1.com:9042"
+ * "100.66.0.0/15" = "cassandra.datacenter2.com"
+ * # IPv6 example:
+ * # "::ffff:6440:0/111" = "cassandra.datacenter1.com:9042"
+ * # "::ffff:6442:0/111" = "cassandra.datacenter2.com"
+ * }
+ * </pre>
+ *
+ * If configured without port, the default 9042 will be used. Also supports IPv6 subnets. Note:
+ * subnets must be represented as prefix blocks, see {@link inet.ipaddr.Address#isPrefixBlock()}.
+ */
+ public static final String ADDRESS_TRANSLATOR_SUBNET_ADDRESSES =
+ "advanced.address-translator.subnet-addresses";
+
+ /**
+ * A default address to fallback to if Cassandra node IP isn't contained in any of the configured
+ * subnets. If configured without port, the default 9042 will be used. Also supports IPv6
+ * addresses.
+ */
+ public static final String ADDRESS_TRANSLATOR_DEFAULT_ADDRESS =
+ "advanced.address-translator.default-address";
+
+ public static DriverOption ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_SUBNET_ADDRESSES;
+ }
+ };
+
+ public static DriverOption ADDRESS_TRANSLATOR_DEFAULT_ADDRESS_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_DEFAULT_ADDRESS;
+ }
+ };
+
+ private static final String DELIMITER = ":";
+ private static final int DEFAULT_PORT = 9042;
+
+ private final List<SubnetAddress> subnetAddresses;
+ private final Optional<InetSocketAddress> defaultAddress;
+ private final String logPrefix;
+
+ public SubnetAddressTranslator(@NonNull DriverContext context) {
+ logPrefix = context.getSessionName();
+ this.subnetAddresses =
+ context.getConfig().getDefaultProfile()
+ .getStringMap(ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION).entrySet().stream()
+ .map(
+ e -> {
+ // Quoted and/or containing forward slashes map keys in reference.conf are read to
+ // strings with additional quotes, eg. 100.64.0.0/15 -> '100.64.0."0/15"' or
+ // "100.64.0.0/15" -> '"100.64.0.0/15"'
Review Comment:
This is a bit of a weird idiosyncrasy of the parser, but like you said, it's invisible to the user, so I think it's ok.
##########
core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/SubnetAddressTranslator.java:
##########
@@ -0,0 +1,226 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.datastax.oss.driver.internal.core.addresstranslation;
+
+import com.datastax.oss.driver.api.core.addresstranslation.AddressTranslator;
+import com.datastax.oss.driver.api.core.config.DriverOption;
+import com.datastax.oss.driver.api.core.context.DriverContext;
+import com.google.common.base.Splitter;
+import edu.umd.cs.findbugs.annotations.NonNull;
+import inet.ipaddr.IPAddress;
+import inet.ipaddr.IPAddressString;
+import java.net.InetSocketAddress;
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.stream.Collectors;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This translator returns the proxy address of the private subnet containing the Cassandra node IP,
+ * or default address if no matching subnets, or passes through the original node address if no
+ * default configured.
+ *
+ * <p>The translator can be used for scenarios when all nodes are behind some kind of proxy, and
+ * that proxy is different for nodes located in different subnets (eg. when Cassandra is deployed in
+ * multiple datacenters/regions). One can use this, for example, for Cassandra on Kubernetes with
+ * different Cassandra datacenters deployed to different Kubernetes clusters.
+ */
+public class SubnetAddressTranslator implements AddressTranslator {
+
+ private static final Logger LOG = LoggerFactory.getLogger(SubnetAddressTranslator.class);
+
+ /**
+ * A map of Cassandra node subnets (CIDR notations) to target addresses, for example (note quoted
+ * keys):
+ *
+ * <pre>
+ * advanced.address-translator.subnet-addresses {
+ * "100.64.0.0/15" = "cassandra.datacenter1.com:9042"
+ * "100.66.0.0/15" = "cassandra.datacenter2.com"
+ * # IPv6 example:
+ * # "::ffff:6440:0/111" = "cassandra.datacenter1.com:9042"
+ * # "::ffff:6442:0/111" = "cassandra.datacenter2.com"
+ * }
+ * </pre>
+ *
+ * If configured without port, the default 9042 will be used. Also supports IPv6 subnets. Note:
+ * subnets must be represented as prefix blocks, see {@link inet.ipaddr.Address#isPrefixBlock()}.
+ */
+ public static final String ADDRESS_TRANSLATOR_SUBNET_ADDRESSES =
+ "advanced.address-translator.subnet-addresses";
+
+ /**
+ * A default address to fallback to if Cassandra node IP isn't contained in any of the configured
+ * subnets. If configured without port, the default 9042 will be used. Also supports IPv6
+ * addresses.
+ */
+ public static final String ADDRESS_TRANSLATOR_DEFAULT_ADDRESS =
+ "advanced.address-translator.default-address";
+
+ public static DriverOption ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_SUBNET_ADDRESSES;
+ }
+ };
+
+ public static DriverOption ADDRESS_TRANSLATOR_DEFAULT_ADDRESS_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_DEFAULT_ADDRESS;
+ }
+ };
+
+ private static final String DELIMITER = ":";
+ private static final int DEFAULT_PORT = 9042;
+
+ private final List<SubnetAddress> subnetAddresses;
+ private final Optional<InetSocketAddress> defaultAddress;
+ private final String logPrefix;
+
+ public SubnetAddressTranslator(@NonNull DriverContext context) {
+ logPrefix = context.getSessionName();
+ this.subnetAddresses =
+ context.getConfig().getDefaultProfile()
+ .getStringMap(ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION).entrySet().stream()
+ .map(
+ e -> {
+ // Quoted and/or containing forward slashes map keys in reference.conf are read to
+ // strings with additional quotes, eg. 100.64.0.0/15 -> '100.64.0."0/15"' or
+ // "100.64.0.0/15" -> '"100.64.0.0/15"'
+ String subnet = e.getKey().replaceAll("\"", "");
+ String address = e.getValue();
+ return new SubnetAddress(subnet, address);
+ })
+ .collect(Collectors.toList());
+ this.defaultAddress =
+ Optional.ofNullable(
+ context
+ .getConfig()
+ .getDefaultProfile()
+ .getString(ADDRESS_TRANSLATOR_DEFAULT_ADDRESS_OPTION, null))
+ .map(SubnetAddressTranslator::parseAddress);
+ SubnetAddressTranslator.validateSubnetsAreNotOverlapping(this.subnetAddresses);
+ }
+
+ @NonNull
+ @Override
+ public InetSocketAddress translate(@NonNull InetSocketAddress address) {
+ InetSocketAddress translatedAddress = null;
+ for (SubnetAddress subnetAddress : subnetAddresses) {
+ if (subnetAddress.contains(address)) {
+ translatedAddress = subnetAddress.address;
+ }
+ }
+ if (translatedAddress == null && defaultAddress.isPresent()) {
+ translatedAddress = defaultAddress.get();
+ }
+ if (translatedAddress == null) {
+ translatedAddress = address;
+ }
+ LOG.debug("[{}] Resolved {} to {}", logPrefix, address, translatedAddress);
+ return translatedAddress;
+ }
+
+ @Override
+ public void close() {}
+
+ private static InetSocketAddress parseAddress(String address) {
+ List<String> addressTuple = Splitter.onPattern(DELIMITER).splitToList(address);
+ if (addressTuple.size() == 2) {
+ return new InetSocketAddress(addressTuple.get(0), Integer.parseInt(addressTuple.get(1)));
Review Comment:
One thing I just realized is that since this is done on the construction of the address translator, it means that we will resolve an IP address from the host name at instantiation; Alternatively `FixedHostNameAddressTranslator` resolves in `InetSocketAddress.translate(address)` which means it will be resolved on control connection initialization and refreshing the node list.
Both can be problematic if we ever expect DNS changes to point to a new IP. One might have to bounce their app to pick up the new IP(s) from the DNS change if suddenly the existing IP you were using no longer points to your service. In Kubernetes, depending on your service/loadbalancer implementation, I suppose its possible the underlying IP may change possibly, right? I think generally it will be stable/static, but may vary depending on implementation.
The driver has a configuration `advanced.resolve-contact-points` that allows one to change the behavior such that contact points aren't initially resolved, rather they get resolved every time we attempt to create a new connection which works resilient to IP changes.
I think it could be useful to utilize this configuration in such a way:
1. defer parsing into `InetSocketAddress` in `translate` itself, much like `FixedHostNameAddressTranslator` does, store the `String` as is in `SubnetAddress`.
2. Use `ContactPoints.extract(String, boolean)` in `translate` . For `resolve`, pull that value from `advanced.resolve-contact-points`. (we could consider doing the same for `FixedHostNameAddressTranslator`, but don't need to solve on the PR).
This would allow the DNS resolution to always happen whenever a connection is created. Additionally, I believe the `DefaultEndPoint` implementations will continue to be considered equal if the underlying IPs change, which would be nice I suppose.
Thoughts?
##########
manual/core/address_resolution/README.md:
##########
@@ -118,6 +118,52 @@ datastax-java-driver.advanced.address-translator.class = com.mycompany.MyAddress
Note: the contact points provided while creating the `CqlSession` are not translated, only addresses
retrieved from or sent by Cassandra nodes are.
+### Fixed proxy hostname
Review Comment:
Fantastic, thanks for writing up docs on this!
##########
manual/core/address_resolution/README.md:
##########
@@ -118,6 +118,52 @@ datastax-java-driver.advanced.address-translator.class = com.mycompany.MyAddress
Note: the contact points provided while creating the `CqlSession` are not translated, only addresses
retrieved from or sent by Cassandra nodes are.
+### Fixed proxy hostname
+
+If your client applications access Cassandra through some kind of proxy (eg. with AWS PrivateLink when all Cassandra
+nodes are exposed via one hostname pointing to AWS Endpoint), you can configure driver with
+`FixedHostNameAddressTranslator` to always translate all node addresses to that same proxy hostname, no matter what IP
+address a node has but still using its native transport port.
+
+To use it, specify the following in the [configuration](../configuration):
+
+```
+datastax-java-driver.advanced.address-translator.class = FixedHostNameAddressTranslator
+advertised-hostname = proxyhostname
+```
+
+### Fixed proxy hostname per subnet
+
+When running Cassandra in a private network and accessing it from outside of that private network via some kind of
+proxy, we have an option to use `FixedHostNameAddressTranslator`. But for multi-datacenter Cassandra deployments, we
+want to have more control over routing queries to a specific datacenter (eg. for optimizing latencies), which requires
+setting up a separate proxy per datacenter.
+
+Normally, each Cassandra datacenter nodes are deployed to a separate subnet to support internode communications in the
+cluster and avoid IP addresses collisions. So when Cassandra broadcasts its nodes IP addresses, we can determine which
+datacenter that node belongs to by checking its IP address against the given datacenter subnet.
+
+For such scenarios you can use `SubnetAddressTranslator` to translate node IPs to the datacenter proxy address
+associated with it.
+
+To use it, specify the following in the [configuration](../configuration):
+```
+datastax-java-driver.advanced.address-translator {
+ class = SubnetAddressTranslator
+ subnet-addresses {
+ "100.64.0.0/15" = "cassandra.datacenter1.com:9042"
+ "100.66.0.0/15" = "cassandra.datacenter2.com" # port defaults to 9042 if not specified
+ # IPv6 example:
+ # "::ffff:6440:0/111" = "cassandra.datacenter1.com:9042"
+ # "::ffff:6442:0/111" = "cassandra.datacenter2.com" # port defaults to 9042 if not specified
+ }
+ # Optional. When configured, addresses not matching the configured subnets are translated to it. Port defaults to 9042 if not specified.
+ default-address = "cassandra.datacenter1.com:9042"
+}
+```
+
+Such setup is common for running Cassandra on Kubernetes with [k8ssandra](https://docs.k8ssandra.io/).
Review Comment:
It would be good to also note that the optional IPAddress dependency must be explicitly declared in your project to use this translator.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#discussion_r1955038288
##########
core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/SubnetAddressTranslator.java:
##########
@@ -0,0 +1,226 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.datastax.oss.driver.internal.core.addresstranslation;
+
+import com.datastax.oss.driver.api.core.addresstranslation.AddressTranslator;
+import com.datastax.oss.driver.api.core.config.DriverOption;
+import com.datastax.oss.driver.api.core.context.DriverContext;
+import com.google.common.base.Splitter;
+import edu.umd.cs.findbugs.annotations.NonNull;
+import inet.ipaddr.IPAddress;
+import inet.ipaddr.IPAddressString;
+import java.net.InetSocketAddress;
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.stream.Collectors;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This translator returns the proxy address of the private subnet containing the Cassandra node IP,
+ * or default address if no matching subnets, or passes through the original node address if no
+ * default configured.
+ *
+ * <p>The translator can be used for scenarios when all nodes are behind some kind of proxy, and
+ * that proxy is different for nodes located in different subnets (eg. when Cassandra is deployed in
+ * multiple datacenters/regions). One can use this, for example, for Cassandra on Kubernetes with
+ * different Cassandra datacenters deployed to different Kubernetes clusters.
+ */
+public class SubnetAddressTranslator implements AddressTranslator {
+
+ private static final Logger LOG = LoggerFactory.getLogger(SubnetAddressTranslator.class);
+
+ /**
+ * A map of Cassandra node subnets (CIDR notations) to target addresses, for example (note quoted
+ * keys):
+ *
+ * <pre>
+ * advanced.address-translator.subnet-addresses {
+ * "100.64.0.0/15" = "cassandra.datacenter1.com:9042"
+ * "100.66.0.0/15" = "cassandra.datacenter2.com"
+ * # IPv6 example:
+ * # "::ffff:6440:0/111" = "cassandra.datacenter1.com:9042"
+ * # "::ffff:6442:0/111" = "cassandra.datacenter2.com"
+ * }
+ * </pre>
+ *
+ * If configured without port, the default 9042 will be used. Also supports IPv6 subnets. Note:
+ * subnets must be represented as prefix blocks, see {@link inet.ipaddr.Address#isPrefixBlock()}.
+ */
+ public static final String ADDRESS_TRANSLATOR_SUBNET_ADDRESSES =
+ "advanced.address-translator.subnet-addresses";
+
+ /**
+ * A default address to fallback to if Cassandra node IP isn't contained in any of the configured
+ * subnets. If configured without port, the default 9042 will be used. Also supports IPv6
+ * addresses.
+ */
+ public static final String ADDRESS_TRANSLATOR_DEFAULT_ADDRESS =
+ "advanced.address-translator.default-address";
+
+ public static DriverOption ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_SUBNET_ADDRESSES;
+ }
+ };
+
+ public static DriverOption ADDRESS_TRANSLATOR_DEFAULT_ADDRESS_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_DEFAULT_ADDRESS;
+ }
+ };
+
+ private static final String DELIMITER = ":";
+ private static final int DEFAULT_PORT = 9042;
+
+ private final List<SubnetAddress> subnetAddresses;
+ private final Optional<InetSocketAddress> defaultAddress;
+ private final String logPrefix;
+
+ public SubnetAddressTranslator(@NonNull DriverContext context) {
+ logPrefix = context.getSessionName();
+ this.subnetAddresses =
+ context.getConfig().getDefaultProfile()
+ .getStringMap(ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION).entrySet().stream()
+ .map(
+ e -> {
+ // Quoted and/or containing forward slashes map keys in reference.conf are read to
+ // strings with additional quotes, eg. 100.64.0.0/15 -> '100.64.0."0/15"' or
+ // "100.64.0.0/15" -> '"100.64.0.0/15"'
+ String subnet = e.getKey().replaceAll("\"", "");
+ String address = e.getValue();
+ return new SubnetAddress(subnet, address);
+ })
+ .collect(Collectors.toList());
+ this.defaultAddress =
+ Optional.ofNullable(
+ context
+ .getConfig()
+ .getDefaultProfile()
+ .getString(ADDRESS_TRANSLATOR_DEFAULT_ADDRESS_OPTION, null))
+ .map(SubnetAddressTranslator::parseAddress);
+ SubnetAddressTranslator.validateSubnetsAreNotOverlapping(this.subnetAddresses);
+ }
+
+ @NonNull
+ @Override
+ public InetSocketAddress translate(@NonNull InetSocketAddress address) {
+ InetSocketAddress translatedAddress = null;
+ for (SubnetAddress subnetAddress : subnetAddresses) {
+ if (subnetAddress.contains(address)) {
+ translatedAddress = subnetAddress.address;
+ }
+ }
+ if (translatedAddress == null && defaultAddress.isPresent()) {
+ translatedAddress = defaultAddress.get();
+ }
+ if (translatedAddress == null) {
+ translatedAddress = address;
+ }
+ LOG.debug("[{}] Resolved {} to {}", logPrefix, address, translatedAddress);
+ return translatedAddress;
+ }
+
+ @Override
+ public void close() {}
+
+ private static InetSocketAddress parseAddress(String address) {
+ List<String> addressTuple = Splitter.onPattern(DELIMITER).splitToList(address);
+ if (addressTuple.size() == 2) {
+ return new InetSocketAddress(addressTuple.get(0), Integer.parseInt(addressTuple.get(1)));
Review Comment:
Just looking now and I realized I gave faulty advice about where to create the `InetSocketAddress`, sorry about that!
I didn't consider the case where `resolve-contact-points` is true. By moving the `extract` call to `translate` it will needlessly resolve the DNS even if `resolve-contact-points` is true every time `translate` is called which we probably don't want in that case. So I think you had it right initially by creating the `InetSocketAddress` in the constructor.
I think what we want to do is to call `extract` in the constructor, as that would resolve the address only once overall (instead of for each endpoint in translate).
This also addresses another concern I had which is that `AddressUtils.resolve` throws a RuntimeException if the DNS can't be resolved, by moving the `extract` call back in to the constructor, I think this would be ok, as it will fail fast.
For the cases where `resolve-contact-points` is false, we'll still resolve every time we attempt to connect which I think we want.
I think I agree with you that maybe overloading the `resolve-contact-points` property feels wrong and creating a separate property under `advanced.address-translator` is the right thing to do.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "jahstreet (via GitHub)" <gi...@apache.org>.
jahstreet commented on code in PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#discussion_r1957162913
##########
core/src/main/resources/reference.conf:
##########
@@ -1020,17 +1020,33 @@ datastax-java-driver {
# the package com.datastax.oss.driver.internal.core.addresstranslation.
#
# The driver provides the following implementations out of the box:
- # - PassThroughAddressTranslator: returns all addresses unchanged
+ # - PassThroughAddressTranslator: returns all addresses unchanged.
# - FixedHostNameAddressTranslator: translates all addresses to a specific hostname.
+ # - SubnetAddressTranslator: translates addresses to hostname based on the subnet match.
# - Ec2MultiRegionAddressTranslator: suitable for an Amazon multi-region EC2 deployment where
# clients are also deployed in EC2. It optimizes network costs by favoring private IPs over
# public ones whenever possible.
#
# You can also specify a custom class that implements AddressTranslator and has a public
# constructor with a DriverContext argument.
class = PassThroughAddressTranslator
+ #
# This property has to be set only in case you use FixedHostNameAddressTranslator.
# advertised-hostname = mycustomhostname
+ #
+ # These properties are only applicable in case you use SubnetAddressTranslator.
+ # subnet-addresses {
+ # "100.64.0.0/15" = "cassandra.datacenter1.com:9042"
+ # "100.66.0.0/15" = "cassandra.datacenter2.com:9042"
+ # # IPv6 example:
+ # # "::ffff:6440:0/111" = "cassandra.datacenter1.com:9042"
+ # # "::ffff:6442:0/111" = "cassandra.datacenter2.com:9042"
+ # }
+ # Optional. When configured, addresses not matching the configured subnets are translated to this address.
+ # default-address = "cassandra.datacenter1.com:9042"
+ # Whether to resolve the addresses once on initialization (if true) or on each node (re-)connection (if false).
+ # If not configured, defaults to false.
+ # resolve-addresses = false
Review Comment:
100%, I expect proxy to have at least 2 replicas eligible for "periodic" restarts, that was the intuition to decide here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "tolbertam (via GitHub)" <gi...@apache.org>.
tolbertam commented on code in PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#discussion_r1956786827
##########
core/src/main/resources/reference.conf:
##########
@@ -1020,17 +1020,33 @@ datastax-java-driver {
# the package com.datastax.oss.driver.internal.core.addresstranslation.
#
# The driver provides the following implementations out of the box:
- # - PassThroughAddressTranslator: returns all addresses unchanged
+ # - PassThroughAddressTranslator: returns all addresses unchanged.
# - FixedHostNameAddressTranslator: translates all addresses to a specific hostname.
+ # - SubnetAddressTranslator: translates addresses to hostname based on the subnet match.
# - Ec2MultiRegionAddressTranslator: suitable for an Amazon multi-region EC2 deployment where
# clients are also deployed in EC2. It optimizes network costs by favoring private IPs over
# public ones whenever possible.
#
# You can also specify a custom class that implements AddressTranslator and has a public
# constructor with a DriverContext argument.
class = PassThroughAddressTranslator
+ #
# This property has to be set only in case you use FixedHostNameAddressTranslator.
# advertised-hostname = mycustomhostname
+ #
+ # These properties are only applicable in case you use SubnetAddressTranslator.
+ # subnet-addresses {
+ # "100.64.0.0/15" = "cassandra.datacenter1.com:9042"
+ # "100.66.0.0/15" = "cassandra.datacenter2.com:9042"
+ # # IPv6 example:
+ # # "::ffff:6440:0/111" = "cassandra.datacenter1.com:9042"
+ # # "::ffff:6442:0/111" = "cassandra.datacenter2.com:9042"
+ # }
+ # Optional. When configured, addresses not matching the configured subnets are translated to this address.
+ # default-address = "cassandra.datacenter1.com:9042"
+ # Whether to resolve the addresses once on initialization (if true) or on each node (re-)connection (if false).
+ # If not configured, defaults to false.
+ # resolve-addresses = false
Review Comment:
:+1: while `resolve-contact-points` defaults to `true`, I was thinking about whether we should do the same here, but I think you probably will generally use a DNS name that might change its backing IPs from time to time, so makes sense for this to be `false`. Was that what you were thinking as well?
##########
core/src/test/java/com/datastax/oss/driver/internal/core/addresstranslation/SubnetAddressTranslatorTest.java:
##########
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.datastax.oss.driver.internal.core.addresstranslation;
+
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatIllegalArgumentException;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
+
+import com.datastax.oss.driver.api.core.config.DriverExecutionProfile;
+import com.datastax.oss.driver.internal.core.context.DefaultDriverContext;
+import com.datastax.oss.driver.internal.core.context.MockedDriverContextFactory;
+import com.google.common.collect.ImmutableMap;
+import java.net.InetSocketAddress;
+import java.util.Map;
+import java.util.Optional;
+import org.junit.Test;
+
+public class SubnetAddressTranslatorTest {
Review Comment:
Mind adding:
```suggestion
@SuppressWarnings("resource")
public class SubnetAddressTranslatorTest {
```
to avoid the warnings around using `new SubnetAddressTranslator` without a try block? Since `AddressTranslator` is `AutoCloseable` it causes IntelliJ to throw some warnings, but I think its fine to use it in this way for the test.
##########
core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/SubnetAddressTranslator.java:
##########
@@ -0,0 +1,235 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.datastax.oss.driver.internal.core.addresstranslation;
+
+import com.datastax.oss.driver.api.core.addresstranslation.AddressTranslator;
+import com.datastax.oss.driver.api.core.config.DriverOption;
+import com.datastax.oss.driver.api.core.context.DriverContext;
+import com.datastax.oss.driver.internal.core.util.AddressUtils;
+import edu.umd.cs.findbugs.annotations.NonNull;
+import edu.umd.cs.findbugs.annotations.Nullable;
+import inet.ipaddr.IPAddress;
+import inet.ipaddr.IPAddressString;
+import java.net.InetSocketAddress;
+import java.util.List;
+import java.util.Optional;
+import java.util.stream.Collectors;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This translator returns the proxy address of the private subnet containing the Cassandra node IP,
+ * or default address if no matching subnets, or passes through the original node address if no
+ * default configured.
+ *
+ * <p>The translator can be used for scenarios when all nodes are behind some kind of proxy, and
+ * that proxy is different for nodes located in different subnets (eg. when Cassandra is deployed in
+ * multiple datacenters/regions). One can use this, for example, for Cassandra on Kubernetes with
+ * different Cassandra datacenters deployed to different Kubernetes clusters.
+ */
+public class SubnetAddressTranslator implements AddressTranslator {
+ private static final Logger LOG = LoggerFactory.getLogger(SubnetAddressTranslator.class);
+
+ /**
+ * A map of Cassandra node subnets (CIDR notations) to target addresses, for example (note quoted
+ * keys):
+ *
+ * <pre>
+ * advanced.address-translator.subnet-addresses {
+ * "100.64.0.0/15" = "cassandra.datacenter1.com:9042"
+ * "100.66.0.0/15" = "cassandra.datacenter2.com:9042"
+ * # IPv6 example:
+ * # "::ffff:6440:0/111" = "cassandra.datacenter1.com:9042"
+ * # "::ffff:6442:0/111" = "cassandra.datacenter2.com:9042"
+ * }
+ * </pre>
+ *
+ * Note: subnets must be represented as prefix blocks, see {@link
+ * inet.ipaddr.Address#isPrefixBlock()}.
+ */
+ public static final String ADDRESS_TRANSLATOR_SUBNET_ADDRESSES =
+ "advanced.address-translator.subnet-addresses";
+
+ /**
+ * A default address to fallback to if Cassandra node IP isn't contained in any of the configured
+ * subnets.
+ */
+ public static final String ADDRESS_TRANSLATOR_DEFAULT_ADDRESS =
+ "advanced.address-translator.default-address";
+
+ /**
+ * Whether to resolve the addresses on initialization (if true) or on each node (re-)connection
+ * (if false). Defaults to false.
+ */
+ public static final String ADDRESS_TRANSLATOR_RESOLVE_ADDRESSES =
+ "advanced.address-translator.resolve-addresses";
+
+ public static DriverOption ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_SUBNET_ADDRESSES;
+ }
+ };
+
+ public static DriverOption ADDRESS_TRANSLATOR_DEFAULT_ADDRESS_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_DEFAULT_ADDRESS;
+ }
+ };
+
+ public static DriverOption ADDRESS_TRANSLATOR_RESOLVE_ADDRESSES_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_RESOLVE_ADDRESSES;
+ }
+ };
+
+ private final List<SubnetAddress> subnetAddresses;
+ private final Optional<InetSocketAddress> defaultAddress;
Review Comment:
mind adding this?
```suggestion
@SuppressWarnings("OptionalUsedAsFieldOrParameterType")
private final Optional<InetSocketAddress> defaultAddress;
```
It bothers me that IntelliJ doesn't look this as I think its completely fine, from [the docs](https://www.jetbrains.com/help/inspectopedia/OptionalUsedAsFieldOrParameterType.html) it looks like the main concern is around `Optional` not being `Serializable`, but that doesn't apply here and I like reasoning about `Optional` types than checking for null.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "jahstreet (via GitHub)" <gi...@apache.org>.
jahstreet commented on code in PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#discussion_r1954008765
##########
core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/SubnetAddressTranslator.java:
##########
@@ -0,0 +1,226 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.datastax.oss.driver.internal.core.addresstranslation;
+
+import com.datastax.oss.driver.api.core.addresstranslation.AddressTranslator;
+import com.datastax.oss.driver.api.core.config.DriverOption;
+import com.datastax.oss.driver.api.core.context.DriverContext;
+import com.google.common.base.Splitter;
+import edu.umd.cs.findbugs.annotations.NonNull;
+import inet.ipaddr.IPAddress;
+import inet.ipaddr.IPAddressString;
+import java.net.InetSocketAddress;
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.stream.Collectors;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This translator returns the proxy address of the private subnet containing the Cassandra node IP,
+ * or default address if no matching subnets, or passes through the original node address if no
+ * default configured.
+ *
+ * <p>The translator can be used for scenarios when all nodes are behind some kind of proxy, and
+ * that proxy is different for nodes located in different subnets (eg. when Cassandra is deployed in
+ * multiple datacenters/regions). One can use this, for example, for Cassandra on Kubernetes with
+ * different Cassandra datacenters deployed to different Kubernetes clusters.
+ */
+public class SubnetAddressTranslator implements AddressTranslator {
+
+ private static final Logger LOG = LoggerFactory.getLogger(SubnetAddressTranslator.class);
+
+ /**
+ * A map of Cassandra node subnets (CIDR notations) to target addresses, for example (note quoted
+ * keys):
+ *
+ * <pre>
+ * advanced.address-translator.subnet-addresses {
+ * "100.64.0.0/15" = "cassandra.datacenter1.com:9042"
+ * "100.66.0.0/15" = "cassandra.datacenter2.com"
+ * # IPv6 example:
+ * # "::ffff:6440:0/111" = "cassandra.datacenter1.com:9042"
+ * # "::ffff:6442:0/111" = "cassandra.datacenter2.com"
+ * }
+ * </pre>
+ *
+ * If configured without port, the default 9042 will be used. Also supports IPv6 subnets. Note:
+ * subnets must be represented as prefix blocks, see {@link inet.ipaddr.Address#isPrefixBlock()}.
+ */
+ public static final String ADDRESS_TRANSLATOR_SUBNET_ADDRESSES =
+ "advanced.address-translator.subnet-addresses";
+
+ /**
+ * A default address to fallback to if Cassandra node IP isn't contained in any of the configured
+ * subnets. If configured without port, the default 9042 will be used. Also supports IPv6
+ * addresses.
+ */
+ public static final String ADDRESS_TRANSLATOR_DEFAULT_ADDRESS =
+ "advanced.address-translator.default-address";
+
+ public static DriverOption ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_SUBNET_ADDRESSES;
+ }
+ };
+
+ public static DriverOption ADDRESS_TRANSLATOR_DEFAULT_ADDRESS_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_DEFAULT_ADDRESS;
+ }
+ };
+
+ private static final String DELIMITER = ":";
+ private static final int DEFAULT_PORT = 9042;
+
+ private final List<SubnetAddress> subnetAddresses;
+ private final Optional<InetSocketAddress> defaultAddress;
+ private final String logPrefix;
+
+ public SubnetAddressTranslator(@NonNull DriverContext context) {
+ logPrefix = context.getSessionName();
+ this.subnetAddresses =
+ context.getConfig().getDefaultProfile()
+ .getStringMap(ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION).entrySet().stream()
+ .map(
+ e -> {
+ // Quoted and/or containing forward slashes map keys in reference.conf are read to
+ // strings with additional quotes, eg. 100.64.0.0/15 -> '100.64.0."0/15"' or
+ // "100.64.0.0/15" -> '"100.64.0.0/15"'
+ String subnet = e.getKey().replaceAll("\"", "");
+ String address = e.getValue();
+ return new SubnetAddress(subnet, address);
+ })
+ .collect(Collectors.toList());
+ this.defaultAddress =
+ Optional.ofNullable(
+ context
+ .getConfig()
+ .getDefaultProfile()
+ .getString(ADDRESS_TRANSLATOR_DEFAULT_ADDRESS_OPTION, null))
+ .map(SubnetAddressTranslator::parseAddress);
+ SubnetAddressTranslator.validateSubnetsAreNotOverlapping(this.subnetAddresses);
+ }
+
+ @NonNull
+ @Override
+ public InetSocketAddress translate(@NonNull InetSocketAddress address) {
+ InetSocketAddress translatedAddress = null;
+ for (SubnetAddress subnetAddress : subnetAddresses) {
+ if (subnetAddress.contains(address)) {
+ translatedAddress = subnetAddress.address;
+ }
+ }
+ if (translatedAddress == null && defaultAddress.isPresent()) {
+ translatedAddress = defaultAddress.get();
+ }
+ if (translatedAddress == null) {
+ translatedAddress = address;
+ }
+ LOG.debug("[{}] Resolved {} to {}", logPrefix, address, translatedAddress);
+ return translatedAddress;
+ }
+
+ @Override
+ public void close() {}
+
+ private static InetSocketAddress parseAddress(String address) {
+ List<String> addressTuple = Splitter.onPattern(DELIMITER).splitToList(address);
+ if (addressTuple.size() == 2) {
+ return new InetSocketAddress(addressTuple.get(0), Integer.parseInt(addressTuple.get(1)));
Review Comment:
I see your point. Indeed, depending on implementation it may happen so. Normally, a proxy (be it on K8s or not) is replicated for HA and replicas indeed can change IPs on restarts/migrations/maintenances or in other use-case-specific scenarios.
I like reusing `ContactPoints#extract`, yes we unobviously couple these classes which asks for moving around utility methods, but feels like a question of preference ;) So I'm good.
If we use `ContactPoints#extract`, then we can get multiple resolved IPs, while `InetSocketAddress#new` always resolves to a first one. I guess that is fine for us to also pick the first one from `ContactPoints#extract` 🤔 .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "aratno (via GitHub)" <gi...@apache.org>.
aratno commented on code in PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#discussion_r1958529763
##########
manual/core/address_resolution/README.md:
##########
@@ -118,6 +118,59 @@ datastax-java-driver.advanced.address-translator.class = com.mycompany.MyAddress
Note: the contact points provided while creating the `CqlSession` are not translated, only addresses
retrieved from or sent by Cassandra nodes are.
+### Fixed proxy hostname
+
+If your client applications access Cassandra through some kind of proxy (eg. with AWS PrivateLink when all Cassandra
+nodes are exposed via one hostname pointing to AWS Endpoint), you can configure driver with
+`FixedHostNameAddressTranslator` to always translate all node addresses to that same proxy hostname, no matter what IP
+address a node has but still using its native transport port.
+
+To use it, specify the following in the [configuration](../configuration):
+
+```
+datastax-java-driver.advanced.address-translator.class = FixedHostNameAddressTranslator
+advertised-hostname = proxyhostname
+```
+
+### Fixed proxy hostname per subnet
+
+When running Cassandra in a private network and accessing it from outside of that private network via some kind of
+proxy, we have an option to use `FixedHostNameAddressTranslator`. But for multi-datacenter Cassandra deployments, we
+want to have more control over routing queries to a specific datacenter (eg. for optimizing latencies), which requires
+setting up a separate proxy per datacenter.
+
+Normally, each Cassandra datacenter nodes are deployed to a different subnet to support internode communications in the
+cluster and avoid IP address collisions. So when Cassandra broadcasts its nodes IP addresses, we can determine which
+datacenter that node belongs to by checking its IP address against the given datacenter subnet.
+
+For such scenarios you can use `SubnetAddressTranslator` to translate node IPs to the datacenter proxy address
+associated with it.
+
+To use it, specify the following in the [configuration](../configuration):
+```
+datastax-java-driver.advanced.address-translator {
+ class = SubnetAddressTranslator
+ subnet-addresses {
+ "100.64.0.0/15" = "cassandra.datacenter1.com:9042"
+ "100.66.0.0/15" = "cassandra.datacenter2.com:9042"
Review Comment:
Thanks for including port configuration here - FixedHostNameAddressTranslator doesn't support ports, so even for single-hostname translation this can be a better option
##########
core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/SubnetAddressTranslator.java:
##########
@@ -0,0 +1,236 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.datastax.oss.driver.internal.core.addresstranslation;
+
+import com.datastax.oss.driver.api.core.addresstranslation.AddressTranslator;
+import com.datastax.oss.driver.api.core.config.DriverOption;
+import com.datastax.oss.driver.api.core.context.DriverContext;
+import com.datastax.oss.driver.internal.core.util.AddressUtils;
+import edu.umd.cs.findbugs.annotations.NonNull;
+import edu.umd.cs.findbugs.annotations.Nullable;
+import inet.ipaddr.IPAddress;
+import inet.ipaddr.IPAddressString;
+import java.net.InetSocketAddress;
+import java.util.List;
+import java.util.Optional;
+import java.util.stream.Collectors;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This translator returns the proxy address of the private subnet containing the Cassandra node IP,
+ * or default address if no matching subnets, or passes through the original node address if no
+ * default configured.
+ *
+ * <p>The translator can be used for scenarios when all nodes are behind some kind of proxy, and
+ * that proxy is different for nodes located in different subnets (eg. when Cassandra is deployed in
+ * multiple datacenters/regions). One can use this, for example, for Cassandra on Kubernetes with
+ * different Cassandra datacenters deployed to different Kubernetes clusters.
+ */
+public class SubnetAddressTranslator implements AddressTranslator {
+ private static final Logger LOG = LoggerFactory.getLogger(SubnetAddressTranslator.class);
+
+ /**
+ * A map of Cassandra node subnets (CIDR notations) to target addresses, for example (note quoted
+ * keys):
+ *
+ * <pre>
+ * advanced.address-translator.subnet-addresses {
+ * "100.64.0.0/15" = "cassandra.datacenter1.com:9042"
+ * "100.66.0.0/15" = "cassandra.datacenter2.com:9042"
+ * # IPv6 example:
+ * # "::ffff:6440:0/111" = "cassandra.datacenter1.com:9042"
+ * # "::ffff:6442:0/111" = "cassandra.datacenter2.com:9042"
+ * }
+ * </pre>
+ *
+ * Note: subnets must be represented as prefix blocks, see {@link
+ * inet.ipaddr.Address#isPrefixBlock()}.
+ */
+ public static final String ADDRESS_TRANSLATOR_SUBNET_ADDRESSES =
+ "advanced.address-translator.subnet-addresses";
+
+ /**
+ * A default address to fallback to if Cassandra node IP isn't contained in any of the configured
+ * subnets.
+ */
+ public static final String ADDRESS_TRANSLATOR_DEFAULT_ADDRESS =
+ "advanced.address-translator.default-address";
+
+ /**
+ * Whether to resolve the addresses on initialization (if true) or on each node (re-)connection
+ * (if false). Defaults to false.
+ */
+ public static final String ADDRESS_TRANSLATOR_RESOLVE_ADDRESSES =
+ "advanced.address-translator.resolve-addresses";
+
+ public static DriverOption ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_SUBNET_ADDRESSES;
+ }
+ };
+
+ public static DriverOption ADDRESS_TRANSLATOR_DEFAULT_ADDRESS_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_DEFAULT_ADDRESS;
+ }
+ };
+
+ public static DriverOption ADDRESS_TRANSLATOR_RESOLVE_ADDRESSES_OPTION =
+ new DriverOption() {
+ @NonNull
+ @Override
+ public String getPath() {
+ return ADDRESS_TRANSLATOR_RESOLVE_ADDRESSES;
+ }
+ };
+
+ private final List<SubnetAddress> subnetAddresses;
+ @SuppressWarnings("OptionalUsedAsFieldOrParameterType")
+ private final Optional<InetSocketAddress> defaultAddress;
+ private final String logPrefix;
+
+ public SubnetAddressTranslator(@NonNull DriverContext context) {
+ logPrefix = context.getSessionName();
+ boolean resolveAddresses =
+ context
+ .getConfig()
+ .getDefaultProfile()
+ .getBoolean(ADDRESS_TRANSLATOR_RESOLVE_ADDRESSES_OPTION, false);
+ this.subnetAddresses =
+ context.getConfig().getDefaultProfile()
+ .getStringMap(ADDRESS_TRANSLATOR_SUBNET_ADDRESSES_OPTION).entrySet().stream()
+ .map(
+ e -> {
+ // Quoted and/or containing forward slashes map keys in reference.conf are read to
+ // strings with additional quotes, eg. 100.64.0.0/15 -> '100.64.0."0/15"' or
+ // "100.64.0.0/15" -> '"100.64.0.0/15"'
+ String subnetCIDR = e.getKey().replaceAll("\"", "");
+ String address = e.getValue();
+ return new SubnetAddress(subnetCIDR, parseAddress(address, resolveAddresses));
+ })
+ .collect(Collectors.toList());
+ this.defaultAddress =
+ Optional.ofNullable(
+ context
+ .getConfig()
+ .getDefaultProfile()
+ .getString(ADDRESS_TRANSLATOR_DEFAULT_ADDRESS_OPTION, null))
+ .map(address -> parseAddress(address, resolveAddresses));
+
+ validateSubnetsAreNotOverlapping(this.subnetAddresses);
+ }
+
+ @NonNull
+ @Override
+ public InetSocketAddress translate(@NonNull InetSocketAddress address) {
+ InetSocketAddress translatedAddress = null;
+ for (SubnetAddress subnetAddress : subnetAddresses) {
+ if (subnetAddress.contains(address)) {
+ translatedAddress = subnetAddress.address;
+ }
+ }
+ if (translatedAddress == null && defaultAddress.isPresent()) {
+ translatedAddress = defaultAddress.get();
+ }
+ if (translatedAddress == null) {
+ translatedAddress = address;
+ }
+ LOG.debug("[{}] Translated {} to {}", logPrefix, address, translatedAddress);
+ return translatedAddress;
+ }
+
+ @Override
+ public void close() {}
+
+ @Nullable
+ private InetSocketAddress parseAddress(String address, boolean resolve) {
+ try {
+ InetSocketAddress parsedAddress = AddressUtils.extract(address, resolve).iterator().next();
+ LOG.debug("[{}] Parsed {} to {}", logPrefix, address, parsedAddress);
+ return parsedAddress;
+ } catch (RuntimeException e) {
+ throw new IllegalArgumentException(
+ String.format("Invalid address %s (%s)", address, e.getMessage()));
Review Comment:
Another exception handling nit - could you attach e as the cause here?
##########
core/src/main/java/com/datastax/oss/driver/internal/core/ContactPoints.java:
##########
@@ -41,7 +38,22 @@ public static Set<EndPoint> merge(
Set<EndPoint> result = Sets.newHashSet(programmaticContactPoints);
for (String spec : configContactPoints) {
- for (InetSocketAddress address : extract(spec, resolve)) {
+
+ Set<InetSocketAddress> addresses = Collections.emptySet();
+ try {
+ addresses = AddressUtils.extract(spec, resolve);
+ } catch (RuntimeException e) {
+ LOG.warn("Ignoring invalid contact point {} ({})", spec, e.getMessage());
Review Comment:
Minor nit, could you include the entire exception here, rather than just the message? Stacktrace could be helpful for debugging.
##########
core/src/main/java/com/datastax/oss/driver/internal/core/util/AddressUtils.java:
##########
@@ -0,0 +1,59 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.datastax.oss.driver.internal.core.util;
+
+import com.datastax.oss.driver.shaded.guava.common.collect.ImmutableSet;
+import java.net.InetAddress;
+import java.net.InetSocketAddress;
+import java.net.UnknownHostException;
+import java.util.HashSet;
+import java.util.Set;
+
+public class AddressUtils {
+
+ public static Set<InetSocketAddress> extract(String address, boolean resolve) {
+ int separator = address.lastIndexOf(':');
+ if (separator < 0) {
+ throw new IllegalArgumentException("expecting format host:port");
+ }
+
+ String host = address.substring(0, separator);
+ String portString = address.substring(separator + 1);
+ int port;
+ try {
+ port = Integer.parseInt(portString);
+ } catch (NumberFormatException e) {
+ throw new IllegalArgumentException("expecting port to be a number, got " + portString, e);
+ }
+ if (!resolve) {
+ return ImmutableSet.of(InetSocketAddress.createUnresolved(host, port));
+ } else {
+ InetAddress[] inetAddresses;
+ try {
+ inetAddresses = InetAddress.getAllByName(host);
+ } catch (UnknownHostException e) {
+ throw new RuntimeException("unknown host " + host, e);
Review Comment:
Really minor nit, but could make the UnknownHostException a cause for the RuntimeException here, without the custom message, the stacktrace should be clear
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "absurdfarce (via GitHub)" <gi...@apache.org>.
absurdfarce commented on code in PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#discussion_r2127402217
##########
core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/Subnet.java:
##########
@@ -0,0 +1,157 @@
+package com.datastax.oss.driver.internal.core.addresstranslation;
+
+import com.datastax.oss.driver.shaded.guava.common.annotations.VisibleForTesting;
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.util.Arrays;
+
+class Subnet {
+ private final byte[] subnet;
+ private final byte[] networkMask;
+ private final byte[] upper;
+ private final byte[] lower;
+
+ private Subnet(byte[] subnet, byte[] networkMask) {
+ this.subnet = subnet;
+ this.networkMask = networkMask;
+
+ byte[] upper = new byte[subnet.length];
+ byte[] lower = new byte[subnet.length];
+ for (int i = 0; i < subnet.length; i++) {
+ upper[i] = (byte) (subnet[i] | ~networkMask[i]);
+ lower[i] = (byte) (subnet[i] & networkMask[i]);
+ }
+ this.upper = upper;
+ this.lower = lower;
+ }
+
+ static Subnet parse(String subnetCIDR) throws UnknownHostException {
+ String[] parts = subnetCIDR.split("/");
Review Comment:
Errorprone complains about this line when building due to [known issues with String.split()](https://errorprone.info/bugpattern/StringSplitter). Recommendation is to replace this with:
```java
List<String> parts = Splitter.on('/').splitToList(subnetCIDR);
```
I used something slightly different to preserve the `String[]` type throughout the rest of the code and it seemed to work fine:
```java
String[] parts = Iterables.toArray(Splitter.on('/').split(subnetCIDR), String.class);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "absurdfarce (via GitHub)" <gi...@apache.org>.
absurdfarce commented on code in PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#discussion_r2127402217
##########
core/src/main/java/com/datastax/oss/driver/internal/core/addresstranslation/Subnet.java:
##########
@@ -0,0 +1,157 @@
+package com.datastax.oss.driver.internal.core.addresstranslation;
+
+import com.datastax.oss.driver.shaded.guava.common.annotations.VisibleForTesting;
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.util.Arrays;
+
+class Subnet {
+ private final byte[] subnet;
+ private final byte[] networkMask;
+ private final byte[] upper;
+ private final byte[] lower;
+
+ private Subnet(byte[] subnet, byte[] networkMask) {
+ this.subnet = subnet;
+ this.networkMask = networkMask;
+
+ byte[] upper = new byte[subnet.length];
+ byte[] lower = new byte[subnet.length];
+ for (int i = 0; i < subnet.length; i++) {
+ upper[i] = (byte) (subnet[i] | ~networkMask[i]);
+ lower[i] = (byte) (subnet[i] & networkMask[i]);
+ }
+ this.upper = upper;
+ this.lower = lower;
+ }
+
+ static Subnet parse(String subnetCIDR) throws UnknownHostException {
+ String[] parts = subnetCIDR.split("/");
Review Comment:
ErrorProne complains about this line when building due to [known issues with String.split()](https://errorprone.info/bugpattern/StringSplitter). Recommendation is to replace this with:
```java
List<String> parts = Splitter.on('/').splitToList(subnetCIDR);
```
I used something slightly different to preserve the `String[]` type throughout the rest of the code and it seemed to work fine:
```java
String[] parts = Iterables.toArray(Splitter.on('/').split(subnetCIDR), String.class);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator to translate Cassandra node IPs from private network based on its subnet mask [cassandra-java-driver]
Posted by "jahstreet (via GitHub)" <gi...@apache.org>.
jahstreet commented on PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#issuecomment-2966014455
Looks good now, but fires some test failures that don't seem related to my changes 🤔 . @absurdfarce are you in the context of these failing tests?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
Re: [PR] Add SubnetAddressTranslator [cassandra-java-driver]
Posted by "absurdfarce (via GitHub)" <gi...@apache.org>.
absurdfarce commented on PR #2013:
URL: https://github.com/apache/cassandra-java-driver/pull/2013#issuecomment-2960430702
Confirmed that the build looks good locally with the recent changes from @jahstreet . Kicking off a DataStax Jenkins run now to confirm that we haven't had any unexpected regressions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org