[jira] [Updated] (AMBARI-25987) Enable Spark to use hadoop native libraries for better performance

["Robin Zuo (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AMBARI-25987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Robin Zuo updated AMBARI-25987:
-------------------------------
    Attachment: image-2023-08-15-15-45-26-664.png

> Enable Spark to use hadoop native libraries for better performance
> ------------------------------------------------------------------
>
>                 Key: AMBARI-25987
>                 URL: https://issues.apache.org/jira/browse/AMBARI-25987
>             Project: Ambari
>          Issue Type: Improvement
>          Components: ambari-server
>            Reporter: Robin Zuo
>            Assignee: Robin Zuo
>            Priority: Major
>         Attachments: image-2023-08-15-15-45-26-664.png
>
>
> Currently, when run spark tasks, spark will report cannot load hadoop native libraries.
> Also, the hadoop short-circuit read feature cannot be applied. For better performance, add hadoop native libraries support.
> Some WARN log as following:
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> 23/08/15 13:43:22 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
> ...
> 23/08/15 13:43:25 INFO Client: Requesting a new application from cluster with 3 NodeManagers
> 23/08/15 13:43:26 WARN DomainSocketFactory: The short-circuit local reads feature cannot be used because libhadoop cannot be loaded.
> 23/08/15 13:43:26 INFO Configuration: resource-types.xml not found
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ambari.apache.org
For additional commands, e-mail: issues-help@ambari.apache.org