You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2025/12/20 00:24:01 UTC
[jira] [Commented] (HDFS-14738) Reading and writing HDFS data through HDFS nfs3 server often getting stuck
[ https://issues.apache.org/jira/browse/HDFS-14738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18046763#comment-18046763 ]
ASF GitHub Bot commented on HDFS-14738:
---------------------------------------
github-actions[bot] commented on PR #1300:
URL: https://github.com/apache/hadoop/pull/1300#issuecomment-3677073552
We're closing this stale PR because it has been open for 100 days with no activity. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you feel like this was a mistake, or you would like to continue working on it, please feel free to re-open it and ask for a committer to remove the stale tag and review again.
Thanks all for your contribution.
> Reading and writing HDFS data through HDFS nfs3 server often getting stuck
> --------------------------------------------------------------------------
>
> Key: HDFS-14738
> URL: https://issues.apache.org/jira/browse/HDFS-14738
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: nfs
> Affects Versions: 2.7.2
> Reporter: liying
> Assignee: liying
> Priority: Major
> Attachments: HDFS-14738.001.patch
>
>
> We know the NFS Gateway supports NFSv3 and allows HDFS to be mounted as part of the client’s local file system.
> I start the portmap and nfs3server in the hadoop node，and use the command （mount -t nfs -o vers=3,proto=tcp,nolock nfs3serverIP:/ /hdfs） to mount.
> It works well。Then i ues this command to mount on many client.
> And i used linux cp to copy files to the mounted dir(/hdfs), found the speed was very slow . There is a lot of information in the log as the follow:
> 2019-08-14 15:36:28,347 INFO org.apache.hadoop.security.ShellBasedIdMapping: Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:37:03,093 INFO org.apache.hadoop.security.ShellBasedIdMapping: Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:37:03,850 INFO org.apache.hadoop.security.ShellBasedIdMapping: Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:37:14,780 INFO org.apache.hadoop.security.ShellBasedIdMapping: Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:37:14,928 INFO org.apache.hadoop.security.ShellBasedIdMapping: Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:37:28,410 INFO org.apache.hadoop.security.ShellBasedIdMapping: Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:38:09,310 INFO org.apache.hadoop.security.ShellBasedIdMapping: Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:38:10,069 INFO org.apache.hadoop.security.ShellBasedIdMapping: Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:38:14,856 INFO org.apache.hadoop.security.ShellBasedIdMapping: Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:38:14,957 INFO org.apache.hadoop.security.ShellBasedIdMapping: Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:38:28,475 INFO org.apache.hadoop.security.ShellBasedIdMapping: Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:39:14,923 INFO org.apache.hadoop.security.ShellBasedIdMapping: Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:39:14,987 INFO org.apache.hadoop.security.ShellBasedIdMapping: Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:39:15,541 INFO org.apache.hadoop.security.ShellBasedIdMapping: Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:39:16,287 INFO org.apache.hadoop.security.ShellBasedIdMapping: Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:39:28,530 INFO org.apache.hadoop.security.ShellBasedIdMapping: Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:40:15,015 INFO org.apache.hadoop.security.ShellBasedIdMapping: Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:40:15,024 INFO org.apache.hadoop.security.ShellBasedIdMapping: Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:40:15,028 INFO org.apache.hadoop.security.ShellBasedIdMapping: Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:40:21,757 INFO org.apache.hadoop.security.ShellBasedIdMapping: Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:40:22,508 INFO org.apache.hadoop.security.ShellBasedIdMapping: Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:40:28,583 INFO org.apache.hadoop.security.ShellBasedIdMapping: Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:41:15,063 INFO org.apache.hadoop.security.ShellBasedIdMapping: Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:41:15,086 INFO org.apache.hadoop.security.ShellBasedIdMapping: Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:41:27,983 INFO org.apache.hadoop.security.ShellBasedIdMapping: Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:41:28,647 INFO org.apache.hadoop.security.ShellBasedIdMapping: Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:41:28,729 INFO org.apache.hadoop.security.ShellBasedIdMapping: Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:42:15,184 INFO org.apache.hadoop.security.ShellBasedIdMapping: Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:42:15,191 INFO org.apache.hadoop.security.ShellBasedIdMapping: Can't map group supergroup. Use its string hashcode:-1710818332
> 2019-08-14 15:42:28,707 INFO org.apache.hadoop.security.ShellBasedIdMapping: Can't map group supergroup. Use its string hashcode:-1710818332
>
> Then i print the jstack log of nfs3 server jvm, and i found that many threads waiting for monitor entry : BLOCKED(on object monitor). After checking the codes of ShellBasedIdMapping, found that It will invoke the updateMapInternal every time if the user or group doesn't exits. And many methods in this class are made with synchronized. So i think we can avoid to invoke updateMapInternal every time.
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org