[jira] [Commented] (HIVE-28287) Attempt make the scratch directory writable before failing

["Ayush Saxena (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-28287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17850794#comment-17850794 ] 

Ayush Saxena commented on HIVE-28287:
-------------------------------------

Committed to master.
Thanx [~simhadri-g] for the review!!!

> Attempt make the scratch directory writable before failing
> ----------------------------------------------------------
>
>                 Key: HIVE-28287
>                 URL: https://issues.apache.org/jira/browse/HIVE-28287
>             Project: Hive
>          Issue Type: Bug
>          Components: Hive
>            Reporter: Agnes Tevesz
>            Assignee: Ayush Saxena
>            Priority: Major
>              Labels: pull-request-available
>
> When hive is starting up it checks the tmp/hive directory privileges. Even if the Azure managed identity have write access on an Azure storage account, the rwx-wx-wx privileges still enforced, so in case rwxr-xr-x is set, the hive process startup will fail. 
> See the logs of a startup that resulted in failure.
> {code}
> WARNING: An illegal reflective access operation has occurred
> WARNING: Illegal reflective access by org.apache.hadoop.hive.common.StringInternUtils (file:/usr/lib/hive/lib/hive-common-3.1.3000.2023.0.16.0-142.jar) to field java.net.URI.string
> WARNING: Please consider reporting this to the maintainers of org.apache.hadoop.hive.common.StringInternUtils
> WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
> WARNING: All illegal access operations will be denied in a future release
> Exception in thread "main" java.lang.RuntimeException: Error applying authorization policy on hive configuration: The dir: /tmp/hive on HDFS should be writable. Current permissions are: rwxr-xr-x
>     at org.apache.hive.service.cli.CLIService.init(CLIService.java:121)
>     at org.apache.hive.service.cli.thrift.EmbeddedThriftBinaryCLIService.init(EmbeddedThriftBinaryCLIService.java:63)
>     at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:357)
>     at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:287)
>     at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107)
>     at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:677)
>     at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:228)
>     at org.apache.hadoop.hive.metastore.tools.schematool.HiveSchemaHelper.getConnectionToMetastore(HiveSchemaHelper.java:88)
>     at org.apache.hadoop.hive.metastore.tools.schematool.HiveSchemaHelper.getConnectionToMetastore(HiveSchemaHelper.java:103)
>     at org.apache.hadoop.hive.metastore.CDHMetaStoreSchemaInfo.getMetaStoreSchemaVersion(CDHMetaStoreSchemaInfo.java:323)
>     at org.apache.hadoop.hive.metastore.tools.schematool.SchemaToolTaskInitOrUpgrade.execute(SchemaToolTaskInitOrUpgrade.java:41)
>     at org.apache.hadoop.hive.metastore.tools.schematool.MetastoreSchemaTool.run(MetastoreSchemaTool.java:482)
>     at org.apache.hive.beeline.schematool.HiveSchemaTool.main(HiveSchemaTool.java:143)
>     at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>     at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     at java.base/java.lang.reflect.Method.invoke(Method.java:566)
>     at org.apache.hadoop.util.RunJar.run(RunJar.java:318)
>     at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
> Caused by: java.lang.RuntimeException: The dir: /tmp/hive on HDFS should be writable. Current permissions are: rwxr-xr-x
>     at org.apache.hadoop.hive.ql.exec.Utilities.ensurePathIsWritable(Utilities.java:5088)
>     at org.apache.hadoop.hive.ql.session.SessionState.createRootHDFSDir(SessionState.java:896)
>     at org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:837)
>     at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:749)
>     at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:708)
>     at org.apache.hive.service.cli.CLIService.applyAuthorizationConfigPolicy(CLIService.java:133)
>     at org.apache.hive.service.cli.CLIService.init(CLIService.java:118)
>     ... 18 more
> Information schema initialization failed!
> + '[' 1 -eq 0 ']'
> + echo 'Information schema initialization failed!'
> + exit 1
> {code}
> To overcome this issue one approach could be to try to set the tmp/hive folder privileges to rwx-wx-wx before failing the startup.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)