You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "ConfX (Jira)" <ji...@apache.org> on 2023/07/14 13:11:00 UTC
[jira] [Updated] (HADOOP-18802) Collision of config key name fs.viewfs.mounttable.default.name.key to other keys that specify the entry point to mount tables
[ https://issues.apache.org/jira/browse/HADOOP-18802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ConfX updated HADOOP-18802:
---------------------------
Description:
h2. What happened:
When manually set {{fs.viewfs.mounttable.default.name.key}} to {{default}} (the same as default value) in HCommon, test {{org.apache.hadoop.fs.viewfs.TestFcMainOperationsLocalFs#testGlobStatusWithMultipleWildCardMatches}} would fail.
But the test can pass if this parameter is not manually set in the configuration file.
h2. Where's the bug:
In the constructor of {{{}InodeTree{}}}, the tree attempts to get all the mount table entry points set by user in the configuration and process them one by one:
{code:java}
for (Entry<String, String> si : config) {
final String key = si.getKey(); if (!key.startsWith(mountTablePrefix)) { continue; } gotMountTableEntry = true; LinkType linkType; String src = key.substring(mountTablePrefix.length()); ...
{code}
Here {{{}mountTablePrefix="fs.viewfs.mounttable.default."{}}}. However, it just so happens that the name of the configuration users use to specify the default mount table is {{{}fs.viewfs.mounttable.default.name.key{}}}. Thus, if a user specifies the default mount table and uses InodeTree the {{name.key}} would be falsely parsed as the entry point to one of the mount tables, which would cause {{InodeTree}} to throw an exception since {{name.key}} is not a valid entry.
h2. Stack trace:
java.lang.RuntimeException: java.io.IOException: ViewFs: Cannot initialize: Invalid entry in Mount table in config: name.keyat org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) at org.apache.hadoop.fs.viewfs.ViewFsTestSetup.setupForViewFsLocalFs(ViewFsTestSetup.java:88) at org.apache.hadoop.fs.viewfs.TestFcMainOperationsLocalFs.setUp(TestFcMainOperationsLocalFs.java:38)
h2. How to reproduce:
(1) Set {{fs.viewfs.mounttable.default.name.key}} to {{default}}
(2) Run test {{org.apache.hadoop.fs.viewfs.TestFcMainOperationsLocalFs#testGlobStatusWithMultipleWildCardMatches}}
You can use the reproduce.sh in the attachment to easily reproduce the bug:
We are happy to provide a patch if this issue is confirmed.
{{}}
was:
h2. What happened:
When manually set {{fs.viewfs.mounttable.default.name.key}} to {{default}} (the same as default value) in HCommon, test {{org.apache.hadoop.fs.viewfs.TestFcMainOperationsLocalFs#testGlobStatusWithMultipleWildCardMatches}} would fail.
But the test can pass if this parameter is not manually set in the configuration file.
h2. Where's the bug:
In the constructor of {{{}InodeTree{}}}, the tree attempts to get all the mount table entry points set by user in the configuration and process them one by one:
for (Entry<String, String> si : config) { final String key = si.getKey(); if (!key.startsWith(mountTablePrefix)) { continue;
} gotMountTableEntry = true; LinkType linkType; String src = key.substring(mountTablePrefix.length());
...
Here {{{}mountTablePrefix="fs.viewfs.mounttable.default."{}}}. However, it just so happens that the name of the configuration users use to specify the default mount table is {{{}fs.viewfs.mounttable.default.name.key{}}}. Thus, if a user specifies the default mount table and uses InodeTree the {{name.key}} would be falsely parsed as the entry point to one of the mount tables, which would cause {{InodeTree}} to throw an exception since {{name.key}} is not a valid entry.
h2. Stack trace:
java.lang.RuntimeException: java.io.IOException: ViewFs: Cannot initialize: Invalid entry in Mount table in config: name.keyat org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) at org.apache.hadoop.fs.viewfs.ViewFsTestSetup.setupForViewFsLocalFs(ViewFsTestSetup.java:88) at org.apache.hadoop.fs.viewfs.TestFcMainOperationsLocalFs.setUp(TestFcMainOperationsLocalFs.java:38)
h2. How to reproduce:
(1) Set {{fs.viewfs.mounttable.default.name.key}} to {{default}}
(2) Run test {{org.apache.hadoop.fs.viewfs.TestFcMainOperationsLocalFs#testGlobStatusWithMultipleWildCardMatches}}
You can use the reproduce.sh in the attachment to easily reproduce the bug:
We are happy to provide a patch if this issue is confirmed.
{{}}
> Collision of config key name fs.viewfs.mounttable.default.name.key to other keys that specify the entry point to mount tables
> -----------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-18802
> URL: https://issues.apache.org/jira/browse/HADOOP-18802
> Project: Hadoop Common
> Issue Type: Bug
> Components: common, conf, fs
> Reporter: ConfX
> Priority: Critical
> Attachments: reproduce.sh
>
>
> h2. What happened:
> When manually set {{fs.viewfs.mounttable.default.name.key}} to {{default}} (the same as default value) in HCommon, test {{org.apache.hadoop.fs.viewfs.TestFcMainOperationsLocalFs#testGlobStatusWithMultipleWildCardMatches}} would fail.
> But the test can pass if this parameter is not manually set in the configuration file.
> h2. Where's the bug:
> In the constructor of {{{}InodeTree{}}}, the tree attempts to get all the mount table entry points set by user in the configuration and process them one by one:
> {code:java}
> for (Entry<String, String> si : config) {
> final String key = si.getKey(); if (!key.startsWith(mountTablePrefix)) { continue; } gotMountTableEntry = true; LinkType linkType; String src = key.substring(mountTablePrefix.length()); ...
> {code}
> Here {{{}mountTablePrefix="fs.viewfs.mounttable.default."{}}}. However, it just so happens that the name of the configuration users use to specify the default mount table is {{{}fs.viewfs.mounttable.default.name.key{}}}. Thus, if a user specifies the default mount table and uses InodeTree the {{name.key}} would be falsely parsed as the entry point to one of the mount tables, which would cause {{InodeTree}} to throw an exception since {{name.key}} is not a valid entry.
> h2. Stack trace:
> java.lang.RuntimeException: java.io.IOException: ViewFs: Cannot initialize: Invalid entry in Mount table in config: name.keyat org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) at org.apache.hadoop.fs.viewfs.ViewFsTestSetup.setupForViewFsLocalFs(ViewFsTestSetup.java:88) at org.apache.hadoop.fs.viewfs.TestFcMainOperationsLocalFs.setUp(TestFcMainOperationsLocalFs.java:38)
> h2. How to reproduce:
> (1) Set {{fs.viewfs.mounttable.default.name.key}} to {{default}}
> (2) Run test {{org.apache.hadoop.fs.viewfs.TestFcMainOperationsLocalFs#testGlobStatusWithMultipleWildCardMatches}}
> You can use the reproduce.sh in the attachment to easily reproduce the bug:
> We are happy to provide a patch if this issue is confirmed.
> {{}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org