You are viewing a plain text version of this content. The canonical link for it is here.
Posted to pr@cassandra.apache.org by GitBox <gi...@apache.org> on 2023/01/09 13:23:57 UTC
[GitHub] [cassandra] smiklosovic opened a new pull request, #2067: CASSANDRA-14361
smiklosovic opened a new pull request, #2067:
URL: https://github.com/apache/cassandra/pull/2067
Thanks for sending a pull request! Here are some tips if you're new here:
* Ensure you have added or run the [appropriate tests](https://cassandra.apache.org/_/development/testing.html) for your PR.
* Be sure to keep the PR description updated to reflect all changes.
* Write your PR title to summarize what this PR proposes.
* If possible, provide a concise example to reproduce the issue for a faster review.
* Read our [contributor guidelines](https://cassandra.apache.org/_/development/index.html)
* If you're making a documentation change, see our [guide to documentation contribution](https://cassandra.apache.org/_/development/documentation.html)
Commit messages should follow the following format:
```
<One sentence description, usually Jira title or CHANGES.txt summary>
<Optional lengthier description (context on patch)>
patch by <Authors>; reviewed by <Reviewers> for CASSANDRA-#####
Co-authored-by: Name1 <email1>
Co-authored-by: Name2 <email2>
```
The [Cassandra Jira](https://issues.apache.org/jira/projects/CASSANDRA/issues/)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
[GitHub] [cassandra] smiklosovic closed pull request #2067: CASSANDRA-14361
Posted by GitBox <gi...@apache.org>.
smiklosovic closed pull request #2067: CASSANDRA-14361
URL: https://github.com/apache/cassandra/pull/2067
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
[GitHub] [cassandra] adelapena commented on a diff in pull request #2067: CASSANDRA-14361
Posted by GitBox <gi...@apache.org>.
adelapena commented on code in PR #2067:
URL: https://github.com/apache/cassandra/pull/2067#discussion_r1067305865
##########
src/java/org/apache/cassandra/config/CassandraRelevantProperties.java:
##########
@@ -331,8 +331,14 @@
*
* If only keyspaces are specified, mutations for all tables in such keyspace will be replayed
* */
- COMMIT_LOG_REPLAY_LIST("cassandra.replayList", null)
+ COMMIT_LOG_REPLAY_LIST("cassandra.replayList", null),
+ /**
+ * The maximum number of seeds returned by a seed provider before emmitting a warning.
+ * A large seed list may impact effectiveness of the third gossip round.
+ * The default used in SimpleSeedProvider is 20.
+ **/
Review Comment:
```suggestion
*/
```
##########
src/java/org/apache/cassandra/locator/SimpleSeedProvider.java:
##########
@@ -46,14 +60,43 @@ public List<InetAddressAndPort> getSeeds()
{
throw new AssertionError(e);
}
- String[] hosts = conf.seed_provider.parameters.get("seeds").split(",", -1);
+
+ assert conf.seed_provider != null : "conf.seed_provider is null!";
+
+ boolean resolveMultipleIps;
+ String[] hosts;
+
+ Map<String, String> parameters = conf.seed_provider.parameters;
+ if (parameters == null)
+ {
+ resolveMultipleIps = true;
+ hosts = defaultSeeds;
+ }
+ else
+ {
+ hosts = parameters.getOrDefault("seeds", defaultSeeds[0]).split(",", -1);
Review Comment:
Nit: `SEEDS` is only seems used by the test, I would use it here.
##########
NEWS.txt:
##########
@@ -110,6 +110,7 @@ New features
- Added new CQL native scalar functions for collections. The new functions are mostly analogous to the existing
aggregation functions, but they operate on the elements of collection columns. The new functions are `map_keys`,
`map_values`, `collection_count`, `collection_min`, `collection_max`, `collection_sum` and `collection_avg`.
+ - SimpleSeedProvider now can resolve multiple IP addresses per DNS record. See CASSANDRA-14361
Review Comment:
```suggestion
- SimpleSeedProvider now can resolve multiple IP addresses per DNS record. See CASSANDRA-14361.
```
##########
test/unit/org/apache/cassandra/locator/SimpleSeedProviderTest.java:
##########
@@ -0,0 +1,163 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.locator;
+
+import java.net.InetAddress;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.function.Supplier;
+
+import org.junit.Test;
+
+import org.apache.cassandra.config.Config;
+import org.apache.cassandra.config.DatabaseDescriptor;
+import org.apache.cassandra.config.ParameterizedClass;
+import org.apache.cassandra.utils.FBUtilities;
+import org.mockito.MockedStatic;
+import org.mockito.Mockito;
+
+import static java.util.Arrays.asList;
+import static java.util.Collections.singletonList;
+import static org.apache.cassandra.locator.SimpleSeedProvider.SEEDS_KEY;
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertTrue;
+
+public class SimpleSeedProviderTest
+{
+ private static final String dnsName1 = "dns-name-1";
+ private static final String dnsName2 = "dns-name-2";
+
+ @Test
+ public void testSeedsResolution() throws Throwable
+ {
+ MockedStatic<InetAddressAndPort> inetAddressAndPortMock = null;
+ MockedStatic<DatabaseDescriptor> descriptorMock = null;
+ MockedStatic<FBUtilities> fbUtilitiesMock = null;
+
+ try
+ {
+ byte[] addressBytes1 = new byte[]{ 127, 0, 0, 1 };
+ byte[] addressBytes2 = new byte[]{ 127, 0, 0, 2 };
+ byte[] addressBytes3 = new byte[]{ 127, 0, 0, 3 };
+ InetAddressAndPort address1 = new InetAddressAndPort(InetAddress.getByAddress(addressBytes1), addressBytes1, 7000);
+ InetAddressAndPort address2 = new InetAddressAndPort(InetAddress.getByAddress(addressBytes2), addressBytes2, 7000);
+ InetAddressAndPort address3 = new InetAddressAndPort(InetAddress.getByAddress(addressBytes3), addressBytes3, 7000);
+
+ inetAddressAndPortMock = Mockito.mockStatic(InetAddressAndPort.class);
+ inetAddressAndPortMock.when(() -> InetAddressAndPort.getByName(dnsName1)).thenReturn(address1);
+ inetAddressAndPortMock.when(() -> InetAddressAndPort.getAllByName(dnsName1)).thenReturn(singletonList(address1));
+ inetAddressAndPortMock.when(() -> InetAddressAndPort.getAllByName(dnsName2)).thenReturn(asList(address2, address3));
+ inetAddressAndPortMock.when(() -> InetAddressAndPort.getByName(dnsName2)).thenReturn(address2);
+
+ fbUtilitiesMock = Mockito.mockStatic(FBUtilities.class);
+ fbUtilitiesMock.when(() -> FBUtilities.getLocalAddressAndPort()).thenReturn(address1);
+
+ descriptorMock = Mockito.mockStatic(DatabaseDescriptor.class);
+
+
+ Map<String, String> seedProviderArgs = new HashMap<>();
+
+ //
+ // case no. 1
+ //
Review Comment:
Nit: I would use more informative comments than case no.1, case no.2. etc. Perhaps something like:
- `dns 1 with multiple ips per record`
- `dns 1 without multiple ips per record`
- `dns 2 with multiple ips per record`
- `dns 2 without multiple ips per record`
##########
test/unit/org/apache/cassandra/locator/SimpleSeedProviderTest.java:
##########
@@ -0,0 +1,163 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.locator;
+
+import java.net.InetAddress;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.function.Supplier;
+
+import org.junit.Test;
+
+import org.apache.cassandra.config.Config;
+import org.apache.cassandra.config.DatabaseDescriptor;
+import org.apache.cassandra.config.ParameterizedClass;
+import org.apache.cassandra.utils.FBUtilities;
+import org.mockito.MockedStatic;
+import org.mockito.Mockito;
+
+import static java.util.Arrays.asList;
+import static java.util.Collections.singletonList;
+import static org.apache.cassandra.locator.SimpleSeedProvider.SEEDS_KEY;
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertTrue;
+
+public class SimpleSeedProviderTest
+{
+ private static final String dnsName1 = "dns-name-1";
+ private static final String dnsName2 = "dns-name-2";
+
+ @Test
+ public void testSeedsResolution() throws Throwable
+ {
+ MockedStatic<InetAddressAndPort> inetAddressAndPortMock = null;
+ MockedStatic<DatabaseDescriptor> descriptorMock = null;
+ MockedStatic<FBUtilities> fbUtilitiesMock = null;
+
+ try
+ {
+ byte[] addressBytes1 = new byte[]{ 127, 0, 0, 1 };
+ byte[] addressBytes2 = new byte[]{ 127, 0, 0, 2 };
+ byte[] addressBytes3 = new byte[]{ 127, 0, 0, 3 };
+ InetAddressAndPort address1 = new InetAddressAndPort(InetAddress.getByAddress(addressBytes1), addressBytes1, 7000);
+ InetAddressAndPort address2 = new InetAddressAndPort(InetAddress.getByAddress(addressBytes2), addressBytes2, 7000);
+ InetAddressAndPort address3 = new InetAddressAndPort(InetAddress.getByAddress(addressBytes3), addressBytes3, 7000);
+
+ inetAddressAndPortMock = Mockito.mockStatic(InetAddressAndPort.class);
+ inetAddressAndPortMock.when(() -> InetAddressAndPort.getByName(dnsName1)).thenReturn(address1);
+ inetAddressAndPortMock.when(() -> InetAddressAndPort.getAllByName(dnsName1)).thenReturn(singletonList(address1));
+ inetAddressAndPortMock.when(() -> InetAddressAndPort.getAllByName(dnsName2)).thenReturn(asList(address2, address3));
+ inetAddressAndPortMock.when(() -> InetAddressAndPort.getByName(dnsName2)).thenReturn(address2);
+
+ fbUtilitiesMock = Mockito.mockStatic(FBUtilities.class);
+ fbUtilitiesMock.when(() -> FBUtilities.getLocalAddressAndPort()).thenReturn(address1);
Review Comment:
```suggestion
fbUtilitiesMock.when(FBUtilities::getLocalAddressAndPort).thenReturn(address1);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
[GitHub] [cassandra] adelapena commented on a diff in pull request #2067: CASSANDRA-14361
Posted by GitBox <gi...@apache.org>.
adelapena commented on code in PR #2067:
URL: https://github.com/apache/cassandra/pull/2067#discussion_r1068002545
##########
src/java/org/apache/cassandra/locator/SimpleSeedProvider.java:
##########
@@ -62,6 +105,12 @@ public List<InetAddressAndPort> getSeeds()
logger.warn("Seed provider couldn't lookup host {}", host);
}
}
+
+ if (seeds.size() >= SEED_COUNT_WARN_THRESHOLD)
+ logger.warn("Seed provider returned more than {} seeds. " +
+ "A large seed list may impact effectiveness of the third gossip round",
+ SEED_COUNT_WARN_THRESHOLD);
Review Comment:
I understand it would be an overkill to use a guardrail for this, given that this isn't a limit for user actions but just a warning for admins.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org
[GitHub] [cassandra] adelapena commented on a diff in pull request #2067: CASSANDRA-14361
Posted by GitBox <gi...@apache.org>.
adelapena commented on code in PR #2067:
URL: https://github.com/apache/cassandra/pull/2067#discussion_r1067317473
##########
test/unit/org/apache/cassandra/locator/SimpleSeedProviderTest.java:
##########
@@ -0,0 +1,163 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.locator;
+
+import java.net.InetAddress;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.function.Supplier;
+
+import org.junit.Test;
+
+import org.apache.cassandra.config.Config;
+import org.apache.cassandra.config.DatabaseDescriptor;
+import org.apache.cassandra.config.ParameterizedClass;
+import org.apache.cassandra.utils.FBUtilities;
+import org.mockito.MockedStatic;
+import org.mockito.Mockito;
+
+import static java.util.Arrays.asList;
+import static java.util.Collections.singletonList;
+import static org.apache.cassandra.locator.SimpleSeedProvider.SEEDS_KEY;
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertTrue;
+
+public class SimpleSeedProviderTest
+{
+ private static final String dnsName1 = "dns-name-1";
+ private static final String dnsName2 = "dns-name-2";
+
+ @Test
+ public void testSeedsResolution() throws Throwable
+ {
+ MockedStatic<InetAddressAndPort> inetAddressAndPortMock = null;
+ MockedStatic<DatabaseDescriptor> descriptorMock = null;
+ MockedStatic<FBUtilities> fbUtilitiesMock = null;
+
+ try
+ {
+ byte[] addressBytes1 = new byte[]{ 127, 0, 0, 1 };
+ byte[] addressBytes2 = new byte[]{ 127, 0, 0, 2 };
+ byte[] addressBytes3 = new byte[]{ 127, 0, 0, 3 };
+ InetAddressAndPort address1 = new InetAddressAndPort(InetAddress.getByAddress(addressBytes1), addressBytes1, 7000);
+ InetAddressAndPort address2 = new InetAddressAndPort(InetAddress.getByAddress(addressBytes2), addressBytes2, 7000);
+ InetAddressAndPort address3 = new InetAddressAndPort(InetAddress.getByAddress(addressBytes3), addressBytes3, 7000);
+
+ inetAddressAndPortMock = Mockito.mockStatic(InetAddressAndPort.class);
+ inetAddressAndPortMock.when(() -> InetAddressAndPort.getByName(dnsName1)).thenReturn(address1);
+ inetAddressAndPortMock.when(() -> InetAddressAndPort.getAllByName(dnsName1)).thenReturn(singletonList(address1));
+ inetAddressAndPortMock.when(() -> InetAddressAndPort.getAllByName(dnsName2)).thenReturn(asList(address2, address3));
+ inetAddressAndPortMock.when(() -> InetAddressAndPort.getByName(dnsName2)).thenReturn(address2);
+
+ fbUtilitiesMock = Mockito.mockStatic(FBUtilities.class);
+ fbUtilitiesMock.when(() -> FBUtilities.getLocalAddressAndPort()).thenReturn(address1);
+
+ descriptorMock = Mockito.mockStatic(DatabaseDescriptor.class);
+
+
+ Map<String, String> seedProviderArgs = new HashMap<>();
+
+ //
+ // case no. 1
+ //
+ seedProviderArgs.put(SEEDS_KEY, dnsName1);
+ // resolve_multiple_ip_addresses_per_dns_record is implicitly true here
+
+ descriptorMock.when(DatabaseDescriptor::loadConfig).thenReturn(getConfig(() -> seedProviderArgs));
+ SimpleSeedProvider provider = new SimpleSeedProvider(null);
+ List<InetAddressAndPort> seeds = provider.getSeeds();
+
+ assertEquals(1, seeds.size());
+ assertEquals(address1, seeds.get(0));
+
+ //
+ // case no. 2
+ //
+ seedProviderArgs.put(SEEDS_KEY, dnsName2);
+ // resolve_multiple_ip_addresses_per_dns_record is implicitly true here
+
+ descriptorMock.when(DatabaseDescriptor::loadConfig).thenReturn(getConfig(() -> seedProviderArgs));
+ provider = new SimpleSeedProvider(null);
+ seeds = provider.getSeeds();
+
+ assertEquals(2, seeds.size());
+ assertTrue(seeds.containsAll(asList(address2, address3)));
+
+ //
+ // case no. 3
+ //
+ seedProviderArgs.put(SEEDS_KEY, dnsName1);
+ seedProviderArgs.put(SimpleSeedProvider.RESOLVE_MULTIPLE_IP_ADDRESSES_PER_DNS_RECORD_KEY, "false");
+
+ descriptorMock.when(DatabaseDescriptor::loadConfig).thenReturn(getConfig(() -> seedProviderArgs));
+ provider = new SimpleSeedProvider(null);
+ seeds = provider.getSeeds();
+
+ assertEquals(1, seeds.size());
+ assertEquals(address1, seeds.get(0));
+
+ //
+ // case no. 4
+ //
+ seedProviderArgs.put(SEEDS_KEY, dnsName2);
+ seedProviderArgs.put(SimpleSeedProvider.RESOLVE_MULTIPLE_IP_ADDRESSES_PER_DNS_RECORD_KEY, "false");
+
+ descriptorMock.when(DatabaseDescriptor::loadConfig).thenReturn(getConfig(() -> seedProviderArgs));
+ provider = new SimpleSeedProvider(null);
+ seeds = provider.getSeeds();
+
+ assertEquals(1, seeds.size());
+ assertEquals(address2, seeds.get(0));
+
+ //
+ // case no. 5
+ //
+ seedProviderArgs.put(SEEDS_KEY, String.format("%s,%s", dnsName1, dnsName2));
+ seedProviderArgs.put(SimpleSeedProvider.RESOLVE_MULTIPLE_IP_ADDRESSES_PER_DNS_RECORD_KEY, "true");
+
+ descriptorMock.when(DatabaseDescriptor::loadConfig).thenReturn(getConfig(() -> seedProviderArgs));
+ provider = new SimpleSeedProvider(null);
+ seeds = provider.getSeeds();
+
+ assertEquals(3, seeds.size());
+ assertTrue(seeds.containsAll(asList(address1, address2, address3)));
+ }
+ finally
+ {
+ if (inetAddressAndPortMock != null)
+ inetAddressAndPortMock.close();
+
+ if (descriptorMock != null)
+ descriptorMock.close();
+
+ if (fbUtilitiesMock != null)
+ fbUtilitiesMock.close();
+ }
+ }
+
+ private Config getConfig(Supplier<Map<String, String>> parametersSupplier)
Review Comment:
Nit: can be `static`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org