You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "OmniaGM (via GitHub)" <gi...@apache.org> on 2024/04/21 12:04:34 UTC
[PR] KAFKA-15853: Move socket configs into org.apache.kafka.network.SocketServerConfigs [kafka]
OmniaGM opened a new pull request, #15772:
URL: https://github.com/apache/kafka/pull/15772
- Move socket configs and docs out of core and into `org.apache.kafka.network.SocketServerConfigs`
- Move default values for socket configs into same class
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-15853: Move socket configs into org.apache.kafka.network.SocketServerConfigs [kafka]
Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #15772:
URL: https://github.com/apache/kafka/pull/15772#issuecomment-2070748890
Should we move following configs to `SocketServerConfigs`?
1. `controller.listener.names`
https://github.com/apache/kafka/blob/59c781415fc37c89aa087d7c2999cec7f82f6188/core/src/main/scala/kafka/server/KafkaConfig.scala#L118
2. `inter.broker.listener.name`
https://github.com/apache/kafka/blob/59c781415fc37c89aa087d7c2999cec7f82f6188/server/src/main/java/org/apache/kafka/server/config/ReplicationConfigs.java#L117
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-15853: Move socket configs into org.apache.kafka.network.SocketServerConfigs [kafka]
Posted by "OmniaGM (via GitHub)" <gi...@apache.org>.
OmniaGM commented on code in PR #15772:
URL: https://github.com/apache/kafka/pull/15772#discussion_r1574952989
##########
server/src/main/java/org/apache/kafka/network/SocketServerConfigs.java:
##########
@@ -0,0 +1,146 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.network;
+
+import org.apache.kafka.common.network.ListenerName;
+import org.apache.kafka.common.security.auth.SecurityProtocol;
+import org.apache.kafka.server.config.ReplicationConfigs;
+import java.util.Arrays;
+import java.util.stream.Collectors;
+
+public class SocketServerConfigs {
+ public static final String LISTENER_SECURITY_PROTOCOL_MAP_CONFIG = "listener.security.protocol.map";
+ public static final String LISTENER_SECURITY_PROTOCOL_MAP_DEFAULT = Arrays.stream(SecurityProtocol.values())
+ .collect(Collectors.toMap(ListenerName::forSecurityProtocol, sp -> sp))
+ .entrySet()
+ .stream()
+ .map(entry -> entry.getKey().value() + ":" + entry.getValue().name())
+ .collect(Collectors.joining(","));
+ public static final String LISTENER_SECURITY_PROTOCOL_MAP_DOC = "Map between listener names and security protocols. This must be defined for " +
+ "the same security protocol to be usable in more than one port or IP. For example, internal and " +
+ "external traffic can be separated even if SSL is required for both. Concretely, the user could define listeners " +
+ "with names INTERNAL and EXTERNAL and this property as: <code>INTERNAL:SSL,EXTERNAL:SSL</code>. As shown, key and value are " +
+ "separated by a colon and map entries are separated by commas. Each listener name should only appear once in the map. " +
+ "Different security (SSL and SASL) settings can be configured for each listener by adding a normalised " +
+ "prefix (the listener name is lowercased) to the config name. For example, to set a different keystore for the " +
+ "INTERNAL listener, a config with name <code>listener.name.internal.ssl.keystore.location</code> would be set. " +
+ "If the config for the listener name is not set, the config will fallback to the generic config (i.e. <code>ssl.keystore.location</code>). " +
+ "Note that in KRaft a default mapping from the listener names defined by <code>controller.listener.names</code> to PLAINTEXT " +
+ "is assumed if no explicit mapping is provided and no other security protocol is in use.";
+
+ public static final String LISTENERS_CONFIG = "listeners";
+ public static final String LISTENERS_DEFAULT = "PLAINTEXT://:9092";
+ public static final String LISTENERS_DOC = "Listener List - Comma-separated list of URIs we will listen on and the listener names." +
+ String.format(" If the listener name is not a security protocol, <code>%s</code> must also be set.%n", LISTENER_SECURITY_PROTOCOL_MAP_CONFIG) +
+ " Listener names and port numbers must be unique unless %n" +
+ " one listener is an IPv4 address and the other listener is %n" +
+ " an IPv6 address (for the same port).%n" +
+ " Specify hostname as 0.0.0.0 to bind to all interfaces.%n" +
+ " Leave hostname empty to bind to default interface.%n" +
+ " Examples of legal listener lists:%n" +
+ " <code>PLAINTEXT://myhost:9092,SSL://:9091</code>%n" +
+ " <code>CLIENT://0.0.0.0:9092,REPLICATION://localhost:9093</code>%n" +
+ " <code>PLAINTEXT://127.0.0.1:9092,SSL://[::1]:9092</code>%n";
+
+ public static final String ADVERTISED_LISTENERS_CONFIG = "advertised.listeners";
+ public static final String ADVERTISED_LISTENERS_DOC = String.format(
+ "Listeners to publish to ZooKeeper for clients to use, if different than the <code>%s</code> config property." +
+ " In IaaS environments, this may need to be different from the interface to which the broker binds." +
+ " If this is not set, the value for <code>%1$1s</code> will be used." +
+ " Unlike <code>%1$1s</code>, it is not valid to advertise the 0.0.0.0 meta-address.%n" +
+ " Also unlike <code>%1$1s</code>, there can be duplicated ports in this property," +
+ " so that one listener can be configured to advertise another listener's address." +
+ " This can be useful in some cases where external load balancers are used.", LISTENERS_CONFIG);
+
+
+ public static final String CONTROL_PLANE_LISTENER_NAME_CONFIG = "control.plane.listener.name";
+ public static final String CONTROL_PLANE_LISTENER_NAME_DOC = String.format(
+ "Name of listener used for communication between controller and brokers. " +
+ String.format("A broker will use the <code>%s</code> to locate the endpoint in %s list, to listen for connections from the controller. ", LISTENERS_CONFIG) +
Review Comment:
should be fixed now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-15853: Move socket configs into org.apache.kafka.network.SocketServerConfigs [kafka]
Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #15772:
URL: https://github.com/apache/kafka/pull/15772#discussion_r1573880096
##########
server/src/main/java/org/apache/kafka/network/SocketServerConfigs.java:
##########
@@ -0,0 +1,146 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.network;
+
+import org.apache.kafka.common.network.ListenerName;
+import org.apache.kafka.common.security.auth.SecurityProtocol;
+import org.apache.kafka.server.config.ReplicationConfigs;
+import java.util.Arrays;
+import java.util.stream.Collectors;
+
+public class SocketServerConfigs {
+ public static final String LISTENER_SECURITY_PROTOCOL_MAP_CONFIG = "listener.security.protocol.map";
+ public static final String LISTENER_SECURITY_PROTOCOL_MAP_DEFAULT = Arrays.stream(SecurityProtocol.values())
+ .collect(Collectors.toMap(ListenerName::forSecurityProtocol, sp -> sp))
+ .entrySet()
+ .stream()
+ .map(entry -> entry.getKey().value() + ":" + entry.getValue().name())
+ .collect(Collectors.joining(","));
+ public static final String LISTENER_SECURITY_PROTOCOL_MAP_DOC = "Map between listener names and security protocols. This must be defined for " +
+ "the same security protocol to be usable in more than one port or IP. For example, internal and " +
+ "external traffic can be separated even if SSL is required for both. Concretely, the user could define listeners " +
+ "with names INTERNAL and EXTERNAL and this property as: <code>INTERNAL:SSL,EXTERNAL:SSL</code>. As shown, key and value are " +
+ "separated by a colon and map entries are separated by commas. Each listener name should only appear once in the map. " +
+ "Different security (SSL and SASL) settings can be configured for each listener by adding a normalised " +
+ "prefix (the listener name is lowercased) to the config name. For example, to set a different keystore for the " +
+ "INTERNAL listener, a config with name <code>listener.name.internal.ssl.keystore.location</code> would be set. " +
+ "If the config for the listener name is not set, the config will fallback to the generic config (i.e. <code>ssl.keystore.location</code>). " +
+ "Note that in KRaft a default mapping from the listener names defined by <code>controller.listener.names</code> to PLAINTEXT " +
+ "is assumed if no explicit mapping is provided and no other security protocol is in use.";
+
+ public static final String LISTENERS_CONFIG = "listeners";
+ public static final String LISTENERS_DEFAULT = "PLAINTEXT://:9092";
+ public static final String LISTENERS_DOC = "Listener List - Comma-separated list of URIs we will listen on and the listener names." +
+ String.format(" If the listener name is not a security protocol, <code>%s</code> must also be set.%n", LISTENER_SECURITY_PROTOCOL_MAP_CONFIG) +
+ " Listener names and port numbers must be unique unless %n" +
+ " one listener is an IPv4 address and the other listener is %n" +
+ " an IPv6 address (for the same port).%n" +
+ " Specify hostname as 0.0.0.0 to bind to all interfaces.%n" +
+ " Leave hostname empty to bind to default interface.%n" +
+ " Examples of legal listener lists:%n" +
+ " <code>PLAINTEXT://myhost:9092,SSL://:9091</code>%n" +
+ " <code>CLIENT://0.0.0.0:9092,REPLICATION://localhost:9093</code>%n" +
+ " <code>PLAINTEXT://127.0.0.1:9092,SSL://[::1]:9092</code>%n";
+
+ public static final String ADVERTISED_LISTENERS_CONFIG = "advertised.listeners";
+ public static final String ADVERTISED_LISTENERS_DOC = String.format(
+ "Listeners to publish to ZooKeeper for clients to use, if different than the <code>%s</code> config property." +
+ " In IaaS environments, this may need to be different from the interface to which the broker binds." +
+ " If this is not set, the value for <code>%1$1s</code> will be used." +
+ " Unlike <code>%1$1s</code>, it is not valid to advertise the 0.0.0.0 meta-address.%n" +
+ " Also unlike <code>%1$1s</code>, there can be duplicated ports in this property," +
+ " so that one listener can be configured to advertise another listener's address." +
+ " This can be useful in some cases where external load balancers are used.", LISTENERS_CONFIG);
+
+
+ public static final String CONTROL_PLANE_LISTENER_NAME_CONFIG = "control.plane.listener.name";
+ public static final String CONTROL_PLANE_LISTENER_NAME_DOC = String.format(
+ "Name of listener used for communication between controller and brokers. " +
+ "A broker will use the <code>%s</code> to locate the endpoint in $ListenersProp list, to listen for connections from the controller. " +
Review Comment:
typo: `$ListenersProp`
##########
core/src/main/scala/kafka/server/KafkaConfig.scala:
##########
@@ -486,20 +400,20 @@ object KafkaConfig {
.define(EarlyStartListenersProp, STRING, null, HIGH, EarlyStartListenersDoc)
/** ********* Socket Server Configuration ***********/
- .define(ListenersProp, STRING, Defaults.LISTENERS, HIGH, ListenersDoc)
- .define(AdvertisedListenersProp, STRING, null, HIGH, AdvertisedListenersDoc)
- .define(ListenerSecurityProtocolMapProp, STRING, Defaults.LISTENER_SECURITY_PROTOCOL_MAP, LOW, ListenerSecurityProtocolMapDoc)
- .define(ControlPlaneListenerNameProp, STRING, null, HIGH, controlPlaneListenerNameDoc)
- .define(SocketSendBufferBytesProp, INT, Defaults.SOCKET_SEND_BUFFER_BYTES, HIGH, SocketSendBufferBytesDoc)
- .define(SocketReceiveBufferBytesProp, INT, Defaults.SOCKET_RECEIVE_BUFFER_BYTES, HIGH, SocketReceiveBufferBytesDoc)
- .define(SocketRequestMaxBytesProp, INT, Defaults.SOCKET_REQUEST_MAX_BYTES, atLeast(1), HIGH, SocketRequestMaxBytesDoc)
- .define(SocketListenBacklogSizeProp, INT, Defaults.SOCKET_LISTEN_BACKLOG_SIZE, atLeast(1), MEDIUM, SocketListenBacklogSizeDoc)
- .define(MaxConnectionsPerIpProp, INT, Defaults.MAX_CONNECTIONS_PER_IP, atLeast(0), MEDIUM, MaxConnectionsPerIpDoc)
- .define(MaxConnectionsPerIpOverridesProp, STRING, Defaults.MAX_CONNECTIONS_PER_IP_OVERRIDES, MEDIUM, MaxConnectionsPerIpOverridesDoc)
- .define(MaxConnectionsProp, INT, Defaults.MAX_CONNECTIONS, atLeast(0), MEDIUM, MaxConnectionsDoc)
- .define(MaxConnectionCreationRateProp, INT, Defaults.MAX_CONNECTION_CREATION_RATE, atLeast(0), MEDIUM, MaxConnectionCreationRateDoc)
- .define(ConnectionsMaxIdleMsProp, LONG, Defaults.CONNECTIONS_MAX_IDLE_MS, MEDIUM, ConnectionsMaxIdleMsDoc)
- .define(FailedAuthenticationDelayMsProp, INT, Defaults.FAILED_AUTHENTICATION_DELAY_MS, atLeast(0), LOW, FailedAuthenticationDelayMsDoc)
+ .define(SocketServerConfigs.LISTENERS_CONFIG, STRING, SocketServerConfigs.LISTENERS_DEFAULT, HIGH, SocketServerConfigs.LISTENERS_DOC)
+ .define(SocketServerConfigs.ADVERTISED_LISTENERS_CONFIG, STRING, null, HIGH, SocketServerConfigs.ADVERTISED_LISTENERS_DOC)
+ .define(SocketServerConfigs.LISTENER_SECURITY_PROTOCOL_MAP_CONFIG, STRING, SocketServerConfigs.LISTENER_SECURITY_PROTOCOL_MAP_DEFAULT, LOW, SocketServerConfigs.LISTENER_SECURITY_PROTOCOL_MAP_DOC)
+ .define(SocketServerConfigs.CONTROL_PLANE_LISTENER_NAME_CONFIG, STRING, null, HIGH, SocketServerConfigs.CONTROL_PLANE_LISTENER_NAME_DOC)
+ .define(SocketServerConfigs.SOCKET_SEND_BUFFER_BYTES_CONFIG, INT, SocketServerConfigs.SOCKET_SEND_BUFFER_BYTES_DEFAULT, HIGH, SocketServerConfigs.SOCKET_SEND_BUFFER_BYTES_DOC)
+ .define(SocketServerConfigs.SOCKET_RECEIVE_BUFFER_BYTES_CONFIG, INT, SocketServerConfigs.SOCKET_RECEIVE_BUFFER_BYTES_DEFAULT, HIGH, SocketServerConfigs.SOCKET_RECEIVE_BUFFER_BYTES_DOC)
+ .define(SocketServerConfigs.SOCKET_REQUEST_MAX_BYTES_CONFIG, INT, SocketServerConfigs.SOCKET_REQUEST_MAX_BYTES_DEFAULT, atLeast(1), HIGH, SocketServerConfigs.SOCKET_REQUEST_MAX_BYTES_DOC)
+ .define(SocketServerConfigs.SOCKET_LISTEN_BACKLOG_SIZE_CONFIG, INT, SocketServerConfigs.SOCKET_LISTEN_BACKLOG_SIZE_DEFAULT, atLeast(1), MEDIUM, SocketServerConfigs.SOCKET_LISTEN_BACKLOG_SIZE_DOC)
+ .define(SocketServerConfigs.MAX_CONNECTIONS_PER_IP_CONFIG, INT, SocketServerConfigs.MAX_CONNECTIONS_PER_IP_DEFAULT, atLeast(0), MEDIUM, SocketServerConfigs.MAX_CONNECTIONS_PER_IP_DOC)
+ .define(SocketServerConfigs.MAX_CONNECTIONS_PER_IP_OVERRIDES_CONFIG, STRING, SocketServerConfigs.MAX_CONNECTIONS_PER_IP_OVERRIDES_DEFAULT, MEDIUM, SocketServerConfigs.MAX_CONNECTIONS_PER_IP_OVERRIDES_DOC)
+ .define(SocketServerConfigs.MAX_CONNECTIONS_CONFIG, INT, SocketServerConfigs.MAX_CONNECTIONS_DEFAULT, atLeast(0), MEDIUM, SocketServerConfigs.MAX_CONNECTIONS_DOC)
+ .define(SocketServerConfigs.MAX_CONNECTION_CREATION_RATE_CONFIG, INT, SocketServerConfigs.MAX_CONNECTION_CREATION_RATE_DEFAULT, atLeast(0), MEDIUM, SocketServerConfigs.MAX_CONNECTION_CREATION_RATE_DOC)
+ .define(SocketServerConfigs.LISTENERS_CONFIG, LONG, SocketServerConfigs.CONNECTIONS_MAX_IDLE_MS_DEFAULT, MEDIUM, SocketServerConfigs.CONNECTIONS_MAX_IDLE_MS_DOC)
Review Comment:
typo: `LISTENERS_CONFIG` -> `CONNECTIONS_MAX_IDLE_MS_CONFIG`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-15853: Move socket configs into org.apache.kafka.network.SocketServerConfigs [kafka]
Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #15772:
URL: https://github.com/apache/kafka/pull/15772#discussion_r1573853257
##########
server/src/main/java/org/apache/kafka/network/SocketServerConfigs.java:
##########
@@ -0,0 +1,146 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.network;
+
+import org.apache.kafka.common.network.ListenerName;
+import org.apache.kafka.common.security.auth.SecurityProtocol;
+import org.apache.kafka.server.config.ReplicationConfigs;
+import java.util.Arrays;
+import java.util.stream.Collectors;
+
+public class SocketServerConfigs {
Review Comment:
> Am not very attached to the name but it made more sense for me during the refactor. WDYT?
that makes sense to me. +1 :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-15853: Move socket configs into org.apache.kafka.network.SocketServerConfigs [kafka]
Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #15772:
URL: https://github.com/apache/kafka/pull/15772#issuecomment-2071861237
Those failed tests pass on my local. will merge it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-15853: Move socket configs into org.apache.kafka.network.SocketServerConfigs [kafka]
Posted by "OmniaGM (via GitHub)" <gi...@apache.org>.
OmniaGM commented on code in PR #15772:
URL: https://github.com/apache/kafka/pull/15772#discussion_r1573851135
##########
server/src/main/java/org/apache/kafka/server/config/Defaults.java:
##########
@@ -46,6 +42,10 @@ public class Defaults {
public static final int METADATA_MAX_IDLE_INTERVAL_MS = 500;
public static final int METADATA_MAX_RETENTION_BYTES = 100 * 1024 * 1024;
public static final boolean DELETE_TOPIC_ENABLE = true;
+ public static final int REQUEST_TIMEOUT_MS = 30000;
+ public static final long CONNECTION_SETUP_TIMEOUT_MS = CommonClientConfigs.DEFAULT_SOCKET_CONNECTION_SETUP_TIMEOUT_MS;
Review Comment:
These will move soon in the final PR couple of prs for moving KafkaConfig out of core. So I would keep it for now until I fully finish this jira. `KafkaConfig.scala` will join the rest of the configs in `server` pretty soon.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-15853: Move socket configs into org.apache.kafka.network.SocketServerConfigs [kafka]
Posted by "OmniaGM (via GitHub)" <gi...@apache.org>.
OmniaGM commented on code in PR #15772:
URL: https://github.com/apache/kafka/pull/15772#discussion_r1573851135
##########
server/src/main/java/org/apache/kafka/server/config/Defaults.java:
##########
@@ -46,6 +42,10 @@ public class Defaults {
public static final int METADATA_MAX_IDLE_INTERVAL_MS = 500;
public static final int METADATA_MAX_RETENTION_BYTES = 100 * 1024 * 1024;
public static final boolean DELETE_TOPIC_ENABLE = true;
+ public static final int REQUEST_TIMEOUT_MS = 30000;
+ public static final long CONNECTION_SETUP_TIMEOUT_MS = CommonClientConfigs.DEFAULT_SOCKET_CONNECTION_SETUP_TIMEOUT_MS;
Review Comment:
These will move soon in the final PR couple of prs for moving KafkaConfig out of core. So I would keep it for now until I fully finish this jira.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-15853: Move socket configs into org.apache.kafka.network.SocketServerConfigs [kafka]
Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #15772:
URL: https://github.com/apache/kafka/pull/15772#issuecomment-2068139760
```
Code Warning
FS Format string should use %n rather than \n in org.apache.kafka.network.SocketServerConfigs.<static initializer for SocketServerConfigs>()
[Bug type VA_FORMAT_STRING_USES_NEWLINE (click for details)](file:///home/chia7712/project/kafka/server/build/reports/spotbugs/main.html#VA_FORMAT_STRING_USES_NEWLINE)
In class org.apache.kafka.network.SocketServerConfigs
In method org.apache.kafka.network.SocketServerConfigs.<static initializer for SocketServerConfigs>()
Called method String.format(String, Object[])
Format string " If the listener name is not a security protocol, <code>%s</code> must also be set.\n"
At SocketServerConfigs.java:[line 48]
FS Format string should use %n rather than \n in org.apache.kafka.network.SocketServerConfigs.<static initializer for SocketServerConfigs>()
[Bug type VA_FORMAT_STRING_USES_NEWLINE (click for details)](file:///home/chia7712/project/kafka/server/build/reports/spotbugs/main.html#VA_FORMAT_STRING_USES_NEWLINE)
In class org.apache.kafka.network.SocketServerConfigs
In method org.apache.kafka.network.SocketServerConfigs.<static initializer for SocketServerConfigs>()
Called method String.format(String, Object[])
Format string "Listeners to publish to ZooKeeper for clients to use, if different than the <code>%s</code> config property. In IaaS environments, this may need to be different from the interface to which the broker binds. If this is not set, the value for <code>%1$1s</code> will be used. Unlike <code>%1$1s</code>, it is not valid to advertise the 0.0.0.0 meta-address.\n Also unlike <code>%1$1s</code>, there can be duplicated ports in this property, so that one listener can be configured to advertise another listener's address. This can be useful in some cases where external load balancers are used."
At SocketServerConfigs.java:[line 60]
FS Format string should use %n rather than \n in org.apache.kafka.network.SocketServerConfigs.<static initializer for SocketServerConfigs>()
[Bug type VA_FORMAT_STRING_USES_NEWLINE (click for details)](file:///home/chia7712/project/kafka/server/build/reports/spotbugs/main.html#VA_FORMAT_STRING_USES_NEWLINE)
In class org.apache.kafka.network.SocketServerConfigs
In method org.apache.kafka.network.SocketServerConfigs.<static initializer for SocketServerConfigs>()
Called method String.format(String, Object[])
Format string "Name of listener used for communication between controller and brokers. A broker will use the <code>%s</code> to locate the endpoint in $ListenersProp list, to listen for connections from the controller. For example, if a broker's config is:\n<code>listeners = INTERNAL://192.1.1.8:9092, EXTERNAL://10.1.1.5:9093, CONTROLLER://192.1.1.8:9094listener.security.protocol.map = INTERNAL:PLAINTEXT, EXTERNAL:SSL, CONTROLLER:SSLcontrol.plane.listener.name = CONTROLLER</code>\nOn startup, the broker will start listening on "192.1.1.8:9094" with security protocol "SSL".\nOn the controller side, when it discovers a broker's published endpoints through ZooKeeper, it will use the <code>%1$1s</code> to find the endpoint, which it will use to establish connection to the broker.\nFor example, if the broker's published endpoints on ZooKeeper are:\n <code>"endpoints" : ["INTERNAL://broker1.example.com:9092","EXTERNAL://broker1.example.com:9093","CONTROLLER://broker1.example.com:9094"]<
/code>\n and the controller's config is:\n<code>listener.security.protocol.map = INTERNAL:PLAINTEXT, EXTERNAL:SSL, CONTROLLER:SSLcontrol.plane.listener.name = CONTROLLER</code>\nthen the controller will use "broker1.example.com:9094" with security protocol "SSL" to connect to the broker.\nIf not explicitly configured, the default value will be null and there will be no dedicated endpoints for controller connections.\nIf explicitly configured, the value cannot be the same as the value of <code>%s</code>."
At SocketServerConfigs.java:[line 71]
```
There are build error.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-15853: Move socket configs into org.apache.kafka.network.SocketServerConfigs [kafka]
Posted by "OmniaGM (via GitHub)" <gi...@apache.org>.
OmniaGM commented on code in PR #15772:
URL: https://github.com/apache/kafka/pull/15772#discussion_r1573897839
##########
server/src/main/java/org/apache/kafka/network/SocketServerConfigs.java:
##########
@@ -0,0 +1,146 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.network;
+
+import org.apache.kafka.common.network.ListenerName;
+import org.apache.kafka.common.security.auth.SecurityProtocol;
+import org.apache.kafka.server.config.ReplicationConfigs;
+import java.util.Arrays;
+import java.util.stream.Collectors;
+
+public class SocketServerConfigs {
+ public static final String LISTENER_SECURITY_PROTOCOL_MAP_CONFIG = "listener.security.protocol.map";
+ public static final String LISTENER_SECURITY_PROTOCOL_MAP_DEFAULT = Arrays.stream(SecurityProtocol.values())
+ .collect(Collectors.toMap(ListenerName::forSecurityProtocol, sp -> sp))
+ .entrySet()
+ .stream()
+ .map(entry -> entry.getKey().value() + ":" + entry.getValue().name())
+ .collect(Collectors.joining(","));
+ public static final String LISTENER_SECURITY_PROTOCOL_MAP_DOC = "Map between listener names and security protocols. This must be defined for " +
+ "the same security protocol to be usable in more than one port or IP. For example, internal and " +
+ "external traffic can be separated even if SSL is required for both. Concretely, the user could define listeners " +
+ "with names INTERNAL and EXTERNAL and this property as: <code>INTERNAL:SSL,EXTERNAL:SSL</code>. As shown, key and value are " +
+ "separated by a colon and map entries are separated by commas. Each listener name should only appear once in the map. " +
+ "Different security (SSL and SASL) settings can be configured for each listener by adding a normalised " +
+ "prefix (the listener name is lowercased) to the config name. For example, to set a different keystore for the " +
+ "INTERNAL listener, a config with name <code>listener.name.internal.ssl.keystore.location</code> would be set. " +
+ "If the config for the listener name is not set, the config will fallback to the generic config (i.e. <code>ssl.keystore.location</code>). " +
+ "Note that in KRaft a default mapping from the listener names defined by <code>controller.listener.names</code> to PLAINTEXT " +
+ "is assumed if no explicit mapping is provided and no other security protocol is in use.";
+
+ public static final String LISTENERS_CONFIG = "listeners";
+ public static final String LISTENERS_DEFAULT = "PLAINTEXT://:9092";
+ public static final String LISTENERS_DOC = "Listener List - Comma-separated list of URIs we will listen on and the listener names." +
+ String.format(" If the listener name is not a security protocol, <code>%s</code> must also be set.%n", LISTENER_SECURITY_PROTOCOL_MAP_CONFIG) +
+ " Listener names and port numbers must be unique unless %n" +
+ " one listener is an IPv4 address and the other listener is %n" +
+ " an IPv6 address (for the same port).%n" +
+ " Specify hostname as 0.0.0.0 to bind to all interfaces.%n" +
+ " Leave hostname empty to bind to default interface.%n" +
+ " Examples of legal listener lists:%n" +
+ " <code>PLAINTEXT://myhost:9092,SSL://:9091</code>%n" +
+ " <code>CLIENT://0.0.0.0:9092,REPLICATION://localhost:9093</code>%n" +
+ " <code>PLAINTEXT://127.0.0.1:9092,SSL://[::1]:9092</code>%n";
+
+ public static final String ADVERTISED_LISTENERS_CONFIG = "advertised.listeners";
+ public static final String ADVERTISED_LISTENERS_DOC = String.format(
+ "Listeners to publish to ZooKeeper for clients to use, if different than the <code>%s</code> config property." +
+ " In IaaS environments, this may need to be different from the interface to which the broker binds." +
+ " If this is not set, the value for <code>%1$1s</code> will be used." +
+ " Unlike <code>%1$1s</code>, it is not valid to advertise the 0.0.0.0 meta-address.%n" +
+ " Also unlike <code>%1$1s</code>, there can be duplicated ports in this property," +
+ " so that one listener can be configured to advertise another listener's address." +
+ " This can be useful in some cases where external load balancers are used.", LISTENERS_CONFIG);
+
+
+ public static final String CONTROL_PLANE_LISTENER_NAME_CONFIG = "control.plane.listener.name";
+ public static final String CONTROL_PLANE_LISTENER_NAME_DOC = String.format(
+ "Name of listener used for communication between controller and brokers. " +
+ "A broker will use the <code>%s</code> to locate the endpoint in $ListenersProp list, to listen for connections from the controller. " +
Review Comment:
good catch published a fix
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-15853: Move socket configs into org.apache.kafka.network.SocketServerConfigs [kafka]
Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 merged PR #15772:
URL: https://github.com/apache/kafka/pull/15772
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-15853: Move socket configs into org.apache.kafka.network.SocketServerConfigs [kafka]
Posted by "OmniaGM (via GitHub)" <gi...@apache.org>.
OmniaGM commented on code in PR #15772:
URL: https://github.com/apache/kafka/pull/15772#discussion_r1573851135
##########
server/src/main/java/org/apache/kafka/server/config/Defaults.java:
##########
@@ -46,6 +42,10 @@ public class Defaults {
public static final int METADATA_MAX_IDLE_INTERVAL_MS = 500;
public static final int METADATA_MAX_RETENTION_BYTES = 100 * 1024 * 1024;
public static final boolean DELETE_TOPIC_ENABLE = true;
+ public static final int REQUEST_TIMEOUT_MS = 30000;
+ public static final long CONNECTION_SETUP_TIMEOUT_MS = CommonClientConfigs.DEFAULT_SOCKET_CONNECTION_SETUP_TIMEOUT_MS;
Review Comment:
These will move soon in the final PR couple of prs for moving KafkaConfig out of core. There will be some refactoring to move more stuff into `CommonClientConfigs` and some to `ServerCommonConfigs`. So I would keep it for now until I fully finish this jira. `KafkaConfig.scala` will join the rest of the configs in `server` pretty soon.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-15853: Move socket configs into org.apache.kafka.network.SocketServerConfigs [kafka]
Posted by "OmniaGM (via GitHub)" <gi...@apache.org>.
OmniaGM commented on code in PR #15772:
URL: https://github.com/apache/kafka/pull/15772#discussion_r1574953974
##########
server/src/main/java/org/apache/kafka/network/SocketServerConfigs.java:
##########
@@ -0,0 +1,146 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.network;
+
+import org.apache.kafka.common.network.ListenerName;
+import org.apache.kafka.common.security.auth.SecurityProtocol;
+import org.apache.kafka.server.config.ReplicationConfigs;
+import java.util.Arrays;
+import java.util.stream.Collectors;
+
+public class SocketServerConfigs {
+ public static final String LISTENER_SECURITY_PROTOCOL_MAP_CONFIG = "listener.security.protocol.map";
+ public static final String LISTENER_SECURITY_PROTOCOL_MAP_DEFAULT = Arrays.stream(SecurityProtocol.values())
+ .collect(Collectors.toMap(ListenerName::forSecurityProtocol, sp -> sp))
+ .entrySet()
+ .stream()
+ .map(entry -> entry.getKey().value() + ":" + entry.getValue().name())
+ .collect(Collectors.joining(","));
+ public static final String LISTENER_SECURITY_PROTOCOL_MAP_DOC = "Map between listener names and security protocols. This must be defined for " +
+ "the same security protocol to be usable in more than one port or IP. For example, internal and " +
+ "external traffic can be separated even if SSL is required for both. Concretely, the user could define listeners " +
+ "with names INTERNAL and EXTERNAL and this property as: <code>INTERNAL:SSL,EXTERNAL:SSL</code>. As shown, key and value are " +
+ "separated by a colon and map entries are separated by commas. Each listener name should only appear once in the map. " +
+ "Different security (SSL and SASL) settings can be configured for each listener by adding a normalised " +
+ "prefix (the listener name is lowercased) to the config name. For example, to set a different keystore for the " +
+ "INTERNAL listener, a config with name <code>listener.name.internal.ssl.keystore.location</code> would be set. " +
+ "If the config for the listener name is not set, the config will fallback to the generic config (i.e. <code>ssl.keystore.location</code>). " +
+ "Note that in KRaft a default mapping from the listener names defined by <code>controller.listener.names</code> to PLAINTEXT " +
+ "is assumed if no explicit mapping is provided and no other security protocol is in use.";
+
+ public static final String LISTENERS_CONFIG = "listeners";
+ public static final String LISTENERS_DEFAULT = "PLAINTEXT://:9092";
+ public static final String LISTENERS_DOC = "Listener List - Comma-separated list of URIs we will listen on and the listener names." +
+ String.format(" If the listener name is not a security protocol, <code>%s</code> must also be set.%n", LISTENER_SECURITY_PROTOCOL_MAP_CONFIG) +
+ " Listener names and port numbers must be unique unless %n" +
+ " one listener is an IPv4 address and the other listener is %n" +
+ " an IPv6 address (for the same port).%n" +
+ " Specify hostname as 0.0.0.0 to bind to all interfaces.%n" +
+ " Leave hostname empty to bind to default interface.%n" +
+ " Examples of legal listener lists:%n" +
+ " <code>PLAINTEXT://myhost:9092,SSL://:9091</code>%n" +
+ " <code>CLIENT://0.0.0.0:9092,REPLICATION://localhost:9093</code>%n" +
+ " <code>PLAINTEXT://127.0.0.1:9092,SSL://[::1]:9092</code>%n";
+
+ public static final String ADVERTISED_LISTENERS_CONFIG = "advertised.listeners";
+ public static final String ADVERTISED_LISTENERS_DOC = String.format(
+ "Listeners to publish to ZooKeeper for clients to use, if different than the <code>%s</code> config property." +
+ " In IaaS environments, this may need to be different from the interface to which the broker binds." +
+ " If this is not set, the value for <code>%1$1s</code> will be used." +
+ " Unlike <code>%1$1s</code>, it is not valid to advertise the 0.0.0.0 meta-address.%n" +
+ " Also unlike <code>%1$1s</code>, there can be duplicated ports in this property," +
+ " so that one listener can be configured to advertise another listener's address." +
+ " This can be useful in some cases where external load balancers are used.", LISTENERS_CONFIG);
+
+
+ public static final String CONTROL_PLANE_LISTENER_NAME_CONFIG = "control.plane.listener.name";
+ public static final String CONTROL_PLANE_LISTENER_NAME_DOC = String.format(
+ "Name of listener used for communication between controller and brokers. " +
+ String.format("A broker will use the <code>%s</code> to locate the endpoint in %s list, to listen for connections from the controller. ", LISTENERS_CONFIG) +
Review Comment:
I also removed the sub `String.format` and used the main one that wrap the whole doc
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-15853: Move socket configs into org.apache.kafka.network.SocketServerConfigs [kafka]
Posted by "OmniaGM (via GitHub)" <gi...@apache.org>.
OmniaGM commented on code in PR #15772:
URL: https://github.com/apache/kafka/pull/15772#discussion_r1573850333
##########
server/src/main/java/org/apache/kafka/network/SocketServerConfigs.java:
##########
@@ -0,0 +1,146 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.network;
+
+import org.apache.kafka.common.network.ListenerName;
+import org.apache.kafka.common.security.auth.SecurityProtocol;
+import org.apache.kafka.server.config.ReplicationConfigs;
+import java.util.Arrays;
+import java.util.stream.Collectors;
+
+public class SocketServerConfigs {
Review Comment:
I name it `SocketServerConfigs ` instead of `ServerSocketConfigs` for the following reasons
1. this is part of `server.network`
2. we have `SocketServe` which will move at some point to same package.
3. java has something with same name which is `java.net.ServerSocket` so we may need to differentiate between classes related `SocketServe` away from build in class.
Am not very attached to the name but it made more sense for me during the refactor. WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-15853: Move socket configs into org.apache.kafka.network.SocketServerConfigs [kafka]
Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #15772:
URL: https://github.com/apache/kafka/pull/15772#issuecomment-2071831061
> controller.listener.names
oh, it will be moved to `KRaftConfigs`. please ignore my previous comments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-15853: Move socket configs into org.apache.kafka.network.SocketServerConfigs [kafka]
Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #15772:
URL: https://github.com/apache/kafka/pull/15772#discussion_r1573848774
##########
server/src/main/java/org/apache/kafka/server/config/Defaults.java:
##########
@@ -46,6 +42,10 @@ public class Defaults {
public static final int METADATA_MAX_IDLE_INTERVAL_MS = 500;
public static final int METADATA_MAX_RETENTION_BYTES = 100 * 1024 * 1024;
public static final boolean DELETE_TOPIC_ENABLE = true;
+ public static final int REQUEST_TIMEOUT_MS = 30000;
+ public static final long CONNECTION_SETUP_TIMEOUT_MS = CommonClientConfigs.DEFAULT_SOCKET_CONNECTION_SETUP_TIMEOUT_MS;
+ public static final long CONNECTION_SETUP_TIMEOUT_MAX_MS = CommonClientConfigs.DEFAULT_SOCKET_CONNECTION_SETUP_TIMEOUT_MAX_MS;
Review Comment:
ditto
##########
server/src/main/java/org/apache/kafka/network/SocketServerConfigs.java:
##########
@@ -0,0 +1,146 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.network;
+
+import org.apache.kafka.common.network.ListenerName;
+import org.apache.kafka.common.security.auth.SecurityProtocol;
+import org.apache.kafka.server.config.ReplicationConfigs;
+import java.util.Arrays;
+import java.util.stream.Collectors;
+
+public class SocketServerConfigs {
+ public static final String LISTENER_SECURITY_PROTOCOL_MAP_CONFIG = "listener.security.protocol.map";
+ public static final String LISTENER_SECURITY_PROTOCOL_MAP_DEFAULT = Arrays.stream(SecurityProtocol.values())
+ .collect(Collectors.toMap(sp -> ListenerName.forSecurityProtocol(sp), sp -> sp))
Review Comment:
`.collect(Collectors.toMap(ListenerName::forSecurityProtocol, sp -> sp))`
##########
server/src/main/java/org/apache/kafka/server/config/Defaults.java:
##########
@@ -46,6 +42,10 @@ public class Defaults {
public static final int METADATA_MAX_IDLE_INTERVAL_MS = 500;
public static final int METADATA_MAX_RETENTION_BYTES = 100 * 1024 * 1024;
public static final boolean DELETE_TOPIC_ENABLE = true;
+ public static final int REQUEST_TIMEOUT_MS = 30000;
+ public static final long CONNECTION_SETUP_TIMEOUT_MS = CommonClientConfigs.DEFAULT_SOCKET_CONNECTION_SETUP_TIMEOUT_MS;
Review Comment:
Do we need `CONNECTION_SETUP_TIMEOUT_MS` as it is used by `KafkaConfig.scala` only.
##########
server/src/main/java/org/apache/kafka/network/SocketServerConfigs.java:
##########
@@ -0,0 +1,146 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.network;
+
+import org.apache.kafka.common.network.ListenerName;
+import org.apache.kafka.common.security.auth.SecurityProtocol;
+import org.apache.kafka.server.config.ReplicationConfigs;
+import java.util.Arrays;
+import java.util.stream.Collectors;
+
+public class SocketServerConfigs {
Review Comment:
Should we call it `ServerSocketConfigs` so as to align it with `ServerLogConfigs`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-15853: Move socket configs into org.apache.kafka.network.SocketServerConfigs [kafka]
Posted by "OmniaGM (via GitHub)" <gi...@apache.org>.
OmniaGM commented on code in PR #15772:
URL: https://github.com/apache/kafka/pull/15772#discussion_r1573850333
##########
server/src/main/java/org/apache/kafka/network/SocketServerConfigs.java:
##########
@@ -0,0 +1,146 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.network;
+
+import org.apache.kafka.common.network.ListenerName;
+import org.apache.kafka.common.security.auth.SecurityProtocol;
+import org.apache.kafka.server.config.ReplicationConfigs;
+import java.util.Arrays;
+import java.util.stream.Collectors;
+
+public class SocketServerConfigs {
Review Comment:
I name it `SocketServerConfigs ` instead of `ServerSocketConfigs` for the following reasons
1. this is part of `server.network`
2. we have `SocketServe` which will move at some point to same package.
3. java has something `java.net.ServerSocket` so we may need to differentiate between classes related `SocketServe` away from build in class.
Am not very attached to the name but it made more sense for me during the refactor. WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-15853: Move socket configs into org.apache.kafka.network.SocketServerConfigs [kafka]
Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #15772:
URL: https://github.com/apache/kafka/pull/15772#issuecomment-2068132025
@OmniaGM I merge #15770 first since it is a smaller PR. Please fix conflicts. thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-15853: Move socket configs into org.apache.kafka.network.SocketServerConfigs [kafka]
Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #15772:
URL: https://github.com/apache/kafka/pull/15772#discussion_r1574671001
##########
server/src/main/java/org/apache/kafka/network/SocketServerConfigs.java:
##########
@@ -0,0 +1,146 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.network;
+
+import org.apache.kafka.common.network.ListenerName;
+import org.apache.kafka.common.security.auth.SecurityProtocol;
+import org.apache.kafka.server.config.ReplicationConfigs;
+import java.util.Arrays;
+import java.util.stream.Collectors;
+
+public class SocketServerConfigs {
+ public static final String LISTENER_SECURITY_PROTOCOL_MAP_CONFIG = "listener.security.protocol.map";
+ public static final String LISTENER_SECURITY_PROTOCOL_MAP_DEFAULT = Arrays.stream(SecurityProtocol.values())
+ .collect(Collectors.toMap(ListenerName::forSecurityProtocol, sp -> sp))
+ .entrySet()
+ .stream()
+ .map(entry -> entry.getKey().value() + ":" + entry.getValue().name())
+ .collect(Collectors.joining(","));
+ public static final String LISTENER_SECURITY_PROTOCOL_MAP_DOC = "Map between listener names and security protocols. This must be defined for " +
+ "the same security protocol to be usable in more than one port or IP. For example, internal and " +
+ "external traffic can be separated even if SSL is required for both. Concretely, the user could define listeners " +
+ "with names INTERNAL and EXTERNAL and this property as: <code>INTERNAL:SSL,EXTERNAL:SSL</code>. As shown, key and value are " +
+ "separated by a colon and map entries are separated by commas. Each listener name should only appear once in the map. " +
+ "Different security (SSL and SASL) settings can be configured for each listener by adding a normalised " +
+ "prefix (the listener name is lowercased) to the config name. For example, to set a different keystore for the " +
+ "INTERNAL listener, a config with name <code>listener.name.internal.ssl.keystore.location</code> would be set. " +
+ "If the config for the listener name is not set, the config will fallback to the generic config (i.e. <code>ssl.keystore.location</code>). " +
+ "Note that in KRaft a default mapping from the listener names defined by <code>controller.listener.names</code> to PLAINTEXT " +
+ "is assumed if no explicit mapping is provided and no other security protocol is in use.";
+
+ public static final String LISTENERS_CONFIG = "listeners";
+ public static final String LISTENERS_DEFAULT = "PLAINTEXT://:9092";
+ public static final String LISTENERS_DOC = "Listener List - Comma-separated list of URIs we will listen on and the listener names." +
+ String.format(" If the listener name is not a security protocol, <code>%s</code> must also be set.%n", LISTENER_SECURITY_PROTOCOL_MAP_CONFIG) +
+ " Listener names and port numbers must be unique unless %n" +
+ " one listener is an IPv4 address and the other listener is %n" +
+ " an IPv6 address (for the same port).%n" +
+ " Specify hostname as 0.0.0.0 to bind to all interfaces.%n" +
+ " Leave hostname empty to bind to default interface.%n" +
+ " Examples of legal listener lists:%n" +
+ " <code>PLAINTEXT://myhost:9092,SSL://:9091</code>%n" +
+ " <code>CLIENT://0.0.0.0:9092,REPLICATION://localhost:9093</code>%n" +
+ " <code>PLAINTEXT://127.0.0.1:9092,SSL://[::1]:9092</code>%n";
+
+ public static final String ADVERTISED_LISTENERS_CONFIG = "advertised.listeners";
+ public static final String ADVERTISED_LISTENERS_DOC = String.format(
+ "Listeners to publish to ZooKeeper for clients to use, if different than the <code>%s</code> config property." +
+ " In IaaS environments, this may need to be different from the interface to which the broker binds." +
+ " If this is not set, the value for <code>%1$1s</code> will be used." +
+ " Unlike <code>%1$1s</code>, it is not valid to advertise the 0.0.0.0 meta-address.%n" +
+ " Also unlike <code>%1$1s</code>, there can be duplicated ports in this property," +
+ " so that one listener can be configured to advertise another listener's address." +
+ " This can be useful in some cases where external load balancers are used.", LISTENERS_CONFIG);
+
+
+ public static final String CONTROL_PLANE_LISTENER_NAME_CONFIG = "control.plane.listener.name";
+ public static final String CONTROL_PLANE_LISTENER_NAME_DOC = String.format(
+ "Name of listener used for communication between controller and brokers. " +
+ String.format("A broker will use the <code>%s</code> to locate the endpoint in %s list, to listen for connections from the controller. ", LISTENERS_CONFIG) +
Review Comment:
It lacks of `CONTROL_PLANE_LISTENER_NAME_CONFIG`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org