You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "pri1712 (via GitHub)" <gi...@apache.org> on 2026/05/14 15:22:47 UTC
[I] HadoopPinotFS initialization crashes with NullPointerException in containerized environments running as unmapped UID [pinot]
pri1712 opened a new issue, #18499:
URL: https://github.com/apache/pinot/issues/18499
**Introduction:**
When running Pinot with the `pinot-hdfs` plugin in Kubernetes, the plugin crashes during startup if the container is running as an unmapped (does not have an entry in `/etc/passwd` ) non-root user (e.g., UID 1001).
The crash happens in HadoopPinotFS.init() when calling FileSystem.get(_hadoopConf). The underlying Hadoop UserGroupInformation (UGI) utilizes Java's UnixLoginModule to look up the OS user. Because the container's UID does not have a corresponding name mapped to it in the /etc/passwd file (which is common in hardened/distroless K8s images), the login module returns null. This causes a fatal NullPointerException and immediately crashes the Pinot component (Controller, Broker, or Server).
**Setup:**
**Pinot Version:** 1.5.0
**Plugin:** `pinot-hdfs`
**Environment:** Kubernetes (running as non-root user 1001 without an /etc/passwd entry)
**Error Logs:**
```
2026/05/14 15:05:44.259 ERROR [PinotFSFactory] [main] Could not instantiate file system for class org.apache.pinot.plugin.filesystem.HadoopPinotFS with scheme hdfs
java.lang.RuntimeException: Could not initialize HadoopPinotFS
at org.apache.pinot.plugin.filesystem.HadoopPinotFS.init(HadoopPinotFS.java:71)
at org.apache.pinot.spi.filesystem.PinotFSFactory.register(PinotFSFactory.java:52)
at org.apache.pinot.spi.filesystem.PinotFSFactory.init(PinotFSFactory.java:72)
at org.apache.pinot.controller.BaseControllerStarter.initPinotFSFactory(BaseControllerStarter.java:871)
at org.apache.pinot.controller.BaseControllerStarter.setUpPinotController(BaseControllerStarter.java:573)
at org.apache.pinot.controller.BaseControllerStarter.start(BaseControllerStarter.java:492)
at org.apache.pinot.tools.service.PinotServiceManager.startController(PinotServiceManager.java:117)
at org.apache.pinot.tools.service.PinotServiceManager.startRole(PinotServiceManager.java:86)
at org.apache.pinot.tools.admin.command.StartServiceManagerCommand.lambda$startBootstrapServices$0(StartServiceManagerCommand.java:241)
at org.apache.pinot.tools.admin.command.StartServiceManagerCommand.startPinotService(StartServiceManagerCommand.java:294)
at org.apache.pinot.tools.admin.command.StartServiceManagerCommand.startBootstrapServices(StartServiceManagerCommand.java:240)
at org.apache.pinot.tools.admin.command.StartServiceManagerCommand.execute(StartServiceManagerCommand.java:183)
at org.apache.pinot.tools.admin.command.StartControllerCommand.execute(StartControllerCommand.java:180)
at org.apache.pinot.tools.Command.call(Command.java:33)
at org.apache.pinot.tools.Command.call(Command.java:29)
at picocli.CommandLine.executeUserObject(CommandLine.java:2031)
at picocli.CommandLine.access$1500(CommandLine.java:148)
at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2469)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2461)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2423)
at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2277)
at picocli.CommandLine$RunLast.execute(CommandLine.java:2425)
at picocli.CommandLine.execute(CommandLine.java:2174)
at org.apache.pinot.tools.admin.PinotAdministrator.execute(PinotAdministrator.java:176)
at org.apache.pinot.tools.admin.PinotAdministrator.main(PinotAdministrator.java:212)
Caused by: java.io.IOException: Invalid UID, could not determine effective user
at org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:2066)
at org.apache.hadoop.security.UserGroupInformation.createLoginUser(UserGroupInformation.java:731)
at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:681)
at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:588)
at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:3888)
at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:3878)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3666)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:557)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:289)
at org.apache.pinot.plugin.filesystem.HadoopPinotFS.init(HadoopPinotFS.java:67)
... 24 more
Caused by: javax.security.auth.login.LoginException: java.lang.NullPointerException: invalid null input: name
at jdk.security.auth/com.sun.security.auth.UnixPrincipal.<init>(UnixPrincipal.java:71)
at jdk.security.auth/com.sun.security.auth.module.UnixLoginModule.login(UnixLoginModule.java:134)
at java.base/javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
at java.base/javax.security.auth.login.LoginContext$4.run(LoginContext.java:679)
at java.base/javax.security.auth.login.LoginContext$4.run(LoginContext.java:677)
at java.base/java.security.AccessController.doPrivileged(AccessController.java:712)
at java.base/javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:677)
at java.base/javax.security.auth.login.LoginContext.login(LoginContext.java:587)
at org.apache.hadoop.security.UserGroupInformation$HadoopLoginContext.login(UserGroupInformation.java:2152)
at org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:2051)
at org.apache.hadoop.security.UserGroupInformation.createLoginUser(UserGroupInformation.java:731)
at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:681)
at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:588)
at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:3888)
at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:3878)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3666)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:557)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:289)
at org.apache.pinot.plugin.filesystem.HadoopPinotFS.init(HadoopPinotFS.java:67)
at org.apache.pinot.spi.filesystem.PinotFSFactory.register(PinotFSFactory.java:52)
at org.apache.pinot.spi.filesystem.PinotFSFactory.init(PinotFSFactory.java:72)
at org.apache.pinot.controller.BaseControllerStarter.initPinotFSFactory(BaseControllerStarter.java:871)
at org.apache.pinot.controller.BaseControllerStarter.setUpPinotController(BaseControllerStarter.java:573)
at org.apache.pinot.controller.BaseControllerStarter.start(BaseControllerStarter.java:492)
at org.apache.pinot.tools.service.PinotServiceManager.startController(PinotServiceManager.java:117)
at org.apache.pinot.tools.service.PinotServiceManager.startRole(PinotServiceManager.java:86)
at org.apache.pinot.tools.admin.command.StartServiceManagerCommand.lambda$startBootstrapServices$0(StartServiceManagerCommand.java:241)
at org.apache.pinot.tools.admin.command.StartServiceManagerCommand.startPinotService(StartServiceManagerCommand.java:294)
at org.apache.pinot.tools.admin.command.StartServiceManagerCommand.startBootstrapServices(StartServiceManagerCommand.java:240)
at org.apache.pinot.tools.admin.command.StartServiceManagerCommand.execute(StartServiceManagerCommand.java:183)
at org.apache.pinot.tools.admin.command.StartControllerCommand.execute(StartControllerCommand.java:180)
at org.apache.pinot.tools.Command.call(Command.java:33)
at org.apache.pinot.tools.Command.call(Command.java:29)
at picocli.CommandLine.executeUserObject(CommandLine.java:2031)
at picocli.CommandLine.access$1500(CommandLine.java:148)
at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2469)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2461)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2423)
at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2277)
at picocli.CommandLine$RunLast.execute(CommandLine.java:2425)
at picocli.CommandLine.execute(CommandLine.java:2174)
at org.apache.pinot.tools.admin.PinotAdministrator.execute(PinotAdministrator.java:176)
at org.apache.pinot.tools.admin.PinotAdministrator.main(PinotAdministrator.java:212)
at java.base/javax.security.auth.login.LoginContext.invoke(LoginContext.java:850)
at java.base/javax.security.auth.login.LoginContext$4.run(LoginContext.java:679)
at java.base/javax.security.auth.login.LoginContext$4.run(LoginContext.java:677)
at java.base/java.security.AccessController.doPrivileged(AccessController.java:712)
at java.base/javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:677)
at java.base/javax.security.auth.login.LoginContext.login(LoginContext.java:587)
at org.apache.hadoop.security.UserGroupInformation$HadoopLoginContext.login(UserGroupInformation.java:2152)
at org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:2051)
... 33 more
2026/05/14 15:05:44.260 ERROR [StartServiceManagerCommand] [main] Failed to start a Pinot [CONTROLLER] at 5.961 since launch
java.lang.RuntimeException: java.lang.RuntimeException: Could not initialize HadoopPinotFS
at org.apache.pinot.spi.filesystem.PinotFSFactory.register(PinotFSFactory.java:56)
at org.apache.pinot.spi.filesystem.PinotFSFactory.init(PinotFSFactory.java:72)
at org.apache.pinot.controller.BaseControllerStarter.initPinotFSFactory(BaseControllerStarter.java:871)
at org.apache.pinot.controller.BaseControllerStarter.setUpPinotController(BaseControllerStarter.java:573)
at org.apache.pinot.controller.BaseControllerStarter.start(BaseControllerStarter.java:492)
at org.apache.pinot.tools.service.PinotServiceManager.startController(PinotServiceManager.java:117)
at org.apache.pinot.tools.service.PinotServiceManager.startRole(PinotServiceManager.java:86)
at org.apache.pinot.tools.admin.command.StartServiceManagerCommand.lambda$startBootstrapServices$0(StartServiceManagerCommand.java:241)
at org.apache.pinot.tools.admin.command.StartServiceManagerCommand.startPinotService(StartServiceManagerCommand.java:294)
at org.apache.pinot.tools.admin.command.StartServiceManagerCommand.startBootstrapServices(StartServiceManagerCommand.java:240)
at org.apache.pinot.tools.admin.command.StartServiceManagerCommand.execute(StartServiceManagerCommand.java:183)
at org.apache.pinot.tools.admin.command.StartControllerCommand.execute(StartControllerCommand.java:180)
at org.apache.pinot.tools.Command.call(Command.java:33)
at org.apache.pinot.tools.Command.call(Command.java:29)
at picocli.CommandLine.executeUserObject(CommandLine.java:2031)
at picocli.CommandLine.access$1500(CommandLine.java:148)
at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2469)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2461)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2423)
at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2277)
at picocli.CommandLine$RunLast.execute(CommandLine.java:2425)
at picocli.CommandLine.execute(CommandLine.java:2174)
at org.apache.pinot.tools.admin.PinotAdministrator.execute(PinotAdministrator.java:176)
at org.apache.pinot.tools.admin.PinotAdministrator.main(PinotAdministrator.java:212)
```
**Expected Behavior/Solutioning:**
It would be better if the plugin could allow users to configure the user via an environment variable, this would ensure that there are no unexpected failures when running containers with non root access permissions.
Also open to any other suggestions to tackle this issue more gracefully!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] HadoopPinotFS initialization crashes with NullPointerException in containerized environments running as unmapped UID [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang closed issue #18499: HadoopPinotFS initialization crashes with NullPointerException in containerized environments running as unmapped UID
URL: https://github.com/apache/pinot/issues/18499
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] HadoopPinotFS initialization crashes with NullPointerException in containerized environments running as unmapped UID [pinot]
Posted by "pri1712 (via GitHub)" <gi...@apache.org>.
pri1712 commented on issue #18499:
URL: https://github.com/apache/pinot/issues/18499#issuecomment-4452103906
I would like to take this up if its deemed to be a a valid issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] HadoopPinotFS initialization crashes with NullPointerException in containerized environments running as unmapped UID [pinot]
Posted by "pri1712 (via GitHub)" <gi...@apache.org>.
pri1712 commented on issue #18499:
URL: https://github.com/apache/pinot/issues/18499#issuecomment-4460533879
https://github.com/apache/pinot/pull/18508
the above fixes this PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org