You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "guangbao zhao (Jira)" <ji...@apache.org> on 2021/09/10 11:03:00 UTC
[jira] [Commented] (HIVE-25474) Concurrency add jars cause
hiveserver2 sys cpu to high
[ https://issues.apache.org/jira/browse/HIVE-25474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17413116#comment-17413116 ]
guangbao zhao commented on HIVE-25474:
--------------------------------------
@[~kkasa] Can help review？thanks
> Concurrency add jars cause hiveserver2 sys cpu to high
> ------------------------------------------------------
>
> Key: HIVE-25474
> URL: https://issues.apache.org/jira/browse/HIVE-25474
> Project: Hive
> Issue Type: Improvement
> Components: Hive, HiveServer2
> Affects Versions: 3.1.2
> Reporter: guangbao zhao
> Assignee: guangbao zhao
> Priority: Major
> Attachments: HIVE-25474.jpg, HIVE-25474.patch, PermissionTest.java
>
>
> In the Linux environment, adding multiple jars concurrently through HiveCli or JDBC will increase the system cpu and even affect the service. Finally, we found that when the add jar is executed, the FileUtil chmod method is used to grant permissions to the downloaded jar file. The performance of this method is very low. So we use the setPosixFilePermissions method of the Files class to test. The performance is seventy to eighty times that of FileUtil (the same file is given permissions in multiple cycles, when it is cycled 1000 times), and as the number of cycles increases, the gap becomes larger and larger. But the file requires jdk7+, which is not friendly to windows. Therefore, if you use the setPosixFilePermissions method of the Files class to grant permissions to files in an operating system that conforms to the posix specification(tested on Mac and Linux), the performance will be improved.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)