[jira] [Commented] (HDFS-17926) Automatically create home directory for users

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HDFS-17926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18083376#comment-18083376 ] 

ASF GitHub Bot commented on HDFS-17926:
---------------------------------------

magnuma3 opened a new pull request, #8514:
URL: https://github.com/apache/hadoop/pull/8514

   <!--
     Thanks for sending a pull request!
       1. If this is your first time, please read our contributor guidelines: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
       2. Make sure your PR title starts with JIRA issue id, e.g., 'HADOOP-17799. Your PR title ...'.
   -->
   
   ### Description of PR
   HDFS-17926
   
   Currently, HDFS does not automatically create a user's home directory (e.g., `/user/<user>`). This requires administrators to manually create home directories, which adds operational overhead and can cause failures for user-facing tools (e.g., MapReduce job submission, Hive, Spark) that assume the home directory exists.
   
   This JIRA tracks the development of automatic home directory creation so that when a user's home directory does not yet exist, HDFS creates it automatically with appropriate ownership (`<username>:<supergroup>`) and permissions (`drwx------`).
   
   **Motivation**:
      - Reduces administrative burden for large multi-tenant clusters
      - Prevents job failures caused by missing home directories
      - Aligns with the behavior expected by higher-level ecosystem tools
    
   
   **Behavior**
     
   When dfs.namenode.auto.create.user.home.enabled=true, the NN intercepts
   the following RPCs and creates the caller's /user/<short-name> if it does
   not yet exist:
     
       create, mkdirs, getListing, getFileInfo, getLocatedFileInfo
   
   `hdfs dfs -ls` and similar commands issue getFileInfo first, so a user's
   home directory is created the first time they touch the cluster.
   
   The directory is created with:
      - owner: caller's short username
      - group: caller's primary group (or a configured group if set)
      - permission: configured octal (default 0700)
      - quota: matched against group/user rules from the new quota config
   
   Creation is performed by the NN superuser (not the requesting user)
   
   Results (success and most failures) are cached by short username so that
   subsequent RPCs of the same user incur only a HashMap lookup (~0.001ms)
   instead of an NN getFileInfo round trip (~0.1ms). 
   
   ### How was this patch tested?
   
   This feature was originally developed and added to an internal fork of Apache Hadoop 3.1.2, and has been running in production for over a year.
   
   ### For code changes:
   
   - [x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
   - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
   - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
   - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files?
   
   ### AI Tooling
   
   If an AI tool was used:
   
   - [ ] The PR includes the phrase "Contains content generated by <tool>"
         where <tool> is the name of the AI tool used.
   - [ ] My use of AI contributions follows the ASF legal policy
         https://www.apache.org/legal/generative-tooling.html




> Automatically create home directory for users
> ---------------------------------------------
>
>                 Key: HDFS-17926
>                 URL: https://issues.apache.org/jira/browse/HDFS-17926
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: namenode
>            Reporter: KWON BYUNGCHANG
>            Priority: Major
>
> Currently, HDFS does not automatically create a user's home directory (e.g., `/user/<user>`). This requires administrators to manually create home directories, which adds operational overhead and can cause failures for user-facing tools (e.g., MapReduce job submission, Hive, Spark) that assume the home directory exists.
> This JIRA tracks the development of automatic home directory creation so that when a user's home directory does not yet exist, HDFS creates it automatically with appropriate ownership (`<username>:<supergroup>`) and permissions (`drwx------`).
> *Motivation:*
>  - Reduces administrative burden for large multi-tenant clusters
>  - Prevents job failures caused by missing home directories
>  - Aligns with the behavior expected by higher-level ecosystem tools
>  
> *Background:*
> This feature was originally developed and added to an internal fork of Apache Hadoop 3.1.2, and has been running in production for over a year. 
> We plan to port the implementation to 3.5.0 and share the patch here.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org