You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-commits@hadoop.apache.org by st...@apache.org on 2022/12/15 11:50:16 UTC
[hadoop] branch branch-3.3.5 updated (c1d18518322 -> 867e530cfd3)
This is an automated email from the ASF dual-hosted git repository.
stevel pushed a change to branch branch-3.3.5
in repository https://gitbox.apache.org/repos/asf/hadoop.git
from c1d18518322 HADOOP-18526. Leak of S3AInstrumentation instances via hadoop Metrics references (#5144)
new a93cca7a700 HADOOP-18574. Changing log level of IOStatistics increment to make the DEBUG logs less noisy (#5223)
new 867e530cfd3 HADOOP-18573. Improve error reporting on non-standard kerberos names (#5221)
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../hadoop/fs/statistics/impl/IOStatisticsStoreImpl.java | 2 +-
.../java/org/apache/hadoop/security/ShellBasedIdMapping.java | 11 +++++++----
.../src/main/java/org/apache/hadoop/util/Shell.java | 3 ++-
3 files changed, 10 insertions(+), 6 deletions(-)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org
[hadoop] 01/02: HADOOP-18574. Changing log level of IOStatistics increment to make the DEBUG logs less noisy (#5223)
Posted by st...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
stevel pushed a commit to branch branch-3.3.5
in repository https://gitbox.apache.org/repos/asf/hadoop.git
commit a93cca7a70065e516ca21436f900ef09cf0132bf
Author: Mehakmeet Singh <me...@gmail.com>
AuthorDate: Thu Dec 15 10:19:18 2022 +0530
HADOOP-18574. Changing log level of IOStatistics increment to make the DEBUG logs less noisy (#5223)
Contributed by: Mehakmeet Singh
---
.../org/apache/hadoop/fs/statistics/impl/IOStatisticsStoreImpl.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/statistics/impl/IOStatisticsStoreImpl.java b/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/statistics/impl/IOStatisticsStoreImpl.java
index 0471703b3b0..6db38208919 100644
--- a/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/statistics/impl/IOStatisticsStoreImpl.java
+++ b/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/statistics/impl/IOStatisticsStoreImpl.java
@@ -190,7 +190,7 @@ final class IOStatisticsStoreImpl extends WrappedIOStatistics
return counter.get();
} else {
long l = incAtomicLong(counter, value);
- LOG.debug("Incrementing counter {} by {} with final value {}",
+ LOG.trace("Incrementing counter {} by {} with final value {}",
key, value, l);
return l;
}
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org
[hadoop] 02/02: HADOOP-18573. Improve error reporting on non-standard kerberos names (#5221)
Posted by st...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
stevel pushed a commit to branch branch-3.3.5
in repository https://gitbox.apache.org/repos/asf/hadoop.git
commit 867e530cfd3cc59948b444848711602cc6e12700
Author: Steve Loughran <st...@cloudera.com>
AuthorDate: Thu Dec 15 11:42:36 2022 +0000
HADOOP-18573. Improve error reporting on non-standard kerberos names (#5221)
The kerberos RPC does not declare any restriction on
characters used in kerberos names, though
implementations MAY be more restrictive.
If the kerberos controller supports use non-conventional
principal names *and the kerberos admin chooses to use them*
this can confuse some of the parsing.
The obvious solution is for the enterprise admins to "not do that"
as a lot of things break, bits of hadoop included.
Harden the hadoop code slightly so at least we fail more gracefully,
so people can then get in touch with their sysadmin and tell them
to stop it.
---
.../java/org/apache/hadoop/security/ShellBasedIdMapping.java | 11 +++++++----
.../src/main/java/org/apache/hadoop/util/Shell.java | 3 ++-
2 files changed, 9 insertions(+), 5 deletions(-)
diff --git a/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/ShellBasedIdMapping.java b/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/ShellBasedIdMapping.java
index d435e480890..d280f75f6ac 100644
--- a/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/ShellBasedIdMapping.java
+++ b/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/ShellBasedIdMapping.java
@@ -37,6 +37,8 @@ import org.apache.hadoop.thirdparty.com.google.common.collect.HashBiMap;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
+import static org.apache.hadoop.util.Shell.bashQuote;
+
/**
* A simple shell-based implementation of {@link IdMappingServiceProvider}
* Map id to user name or group name. It does update every 15 minutes. Only a
@@ -471,26 +473,27 @@ public class ShellBasedIdMapping implements IdMappingServiceProvider {
boolean updated = false;
updateStaticMapping();
+ String name2 = bashQuote(name);
if (OS.startsWith("Linux") || OS.equals("SunOS") || OS.contains("BSD")) {
if (isGrp) {
updated = updateMapInternal(gidNameMap, "group",
- getName2IdCmdNIX(name, true), ":",
+ getName2IdCmdNIX(name2, true), ":",
staticMapping.gidMapping);
} else {
updated = updateMapInternal(uidNameMap, "user",
- getName2IdCmdNIX(name, false), ":",
+ getName2IdCmdNIX(name2, false), ":",
staticMapping.uidMapping);
}
} else {
// Mac
if (isGrp) {
updated = updateMapInternal(gidNameMap, "group",
- getName2IdCmdMac(name, true), "\\s+",
+ getName2IdCmdMac(name2, true), "\\s+",
staticMapping.gidMapping);
} else {
updated = updateMapInternal(uidNameMap, "user",
- getName2IdCmdMac(name, false), "\\s+",
+ getName2IdCmdMac(name2, false), "\\s+",
staticMapping.uidMapping);
}
}
diff --git a/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java b/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java
index f6dcd84e61a..cd36fdbbcce 100644
--- a/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java
+++ b/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java
@@ -145,7 +145,8 @@ public abstract class Shell {
* @param arg the argument to quote
* @return the quoted string
*/
- static String bashQuote(String arg) {
+ @InterfaceAudience.Private
+ public static String bashQuote(String arg) {
StringBuilder buffer = new StringBuilder(arg.length() + 2);
buffer.append('\'')
.append(arg.replace("'", "'\\''"))
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org