You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ambari.apache.org by "Ihor Lukianov (JIRA)" <ji...@apache.org> on 2019/07/16 15:33:00 UTC
[jira] [Commented] (AMBARI-25328) When spark_transport_mode is set
to 'http' then STS server fails to start in Ambari 2.7
[ https://issues.apache.org/jira/browse/AMBARI-25328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16886217#comment-16886217 ]
Ihor Lukianov commented on AMBARI-25328:
----------------------------------------
Spark2 script are already fixed in 2.7.4. Working on fix for zeppelin part
> When spark_transport_mode is set to 'http' then STS server fails to start in Ambari 2.7
> ---------------------------------------------------------------------------------------
>
> Key: AMBARI-25328
> URL: https://issues.apache.org/jira/browse/AMBARI-25328
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
> Affects Versions: 2.7.3
> Reporter: JaySenSharma
> Assignee: Ihor Lukianov
> Priority: Major
> Fix For: 2.7.4
>
> Attachments: Manual_Fix.patch
>
>
> When the "hive.server2.transport.mode" is set tp "http" (instead of "binary") inside the "spark2-hive-site-override" configuration then the "Spark2 Thrift Server" start operation fails via Ambari UI with the following error:
> {code}
> File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 150, in _call_wrapper
> result = _call(command, **kwargs_copy)
> File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 314, in _call
> raise ExecutionFailed(err_msg, code, out, err)
> ExecutionFailed: Execution of '! /usr/hdp/current/spark2-thriftserver/bin/beeline -u 'jdbc:hive2://kerlatest4.example.com:10002/default;transportMode=http' -e '' 2>&1| awk '{print}'|grep -i -e 'Connection refused' -e 'Invalid URL' -e 'Error: Could not open'' returned 1. Error: Could not open client transport with JDBC Uri: jdbc:hive2://kerlatest4.example.com:10002/default;transportMode=http: Could not create http connection to jdbc:hive2://kerlatest4.example.com:10002/default;transportMode=http. HTTP Response code: 405 (state=08S01,code=0)
> Error: Could not open client transport with JDBC Uri: jdbc:hive2://kerlatest4.example.com:10002/default;transportMode=http: Could not create http connection to jdbc:hive2://kerlatest4.example.com:10002/default;transportMode=http. HTTP Response code: 405 (state=08S01,code=0)
> {code}
> This is because the "spark_service.py" script is not appending the "httpPath=cliservice" when the "hive.server2.transport.mode" is set to "http" in the "beeline_url" inside the "/var/lib/ambari-server/resources/stacks/HDP/3.0/services/SPARK2/package/scripts/spark_service.py" script (On Ambari Agent "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/SPARK2/package/scripts/spark_service.py").
> *Manual Workaround:*
> Update the "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/SPARK2/package/scripts/spark_service.py" on agent and amabri server as following with additional If else block to check the spark_transport_mode == 'http':
> {code}
> 146c146,149
> < beeline_url = ["jdbc:hive2://{fqdn}:{spark_thrift_port}/default;principal={hive_kerberos_principal}","transportMode={spark_transport_mode}"]
> ---
> > if params.spark_transport_mode == 'http':
> > beeline_url = ["jdbc:hive2://{fqdn}:{spark_thrift_port}/default;principal={hive_kerberos_principal}","transportMode={spark_transport_mode}","httpPath=cliservice"]
> > else:
> > beeline_url = ["jdbc:hive2://{fqdn}:{spark_thrift_port}/default;principal={hive_kerberos_principal}","transportMode={spark_transport_mode}"]
> 148c151,154
> < beeline_url = ["jdbc:hive2://{fqdn}:{spark_thrift_port}/default","transportMode={spark_transport_mode}"]
> ---
> > if params.spark_transport_mode == 'http':
> > beeline_url = ["jdbc:hive2://{fqdn}:{spark_thrift_port}/default","transportMode={spark_transport_mode}","httpPath=cliservice"]
> > else:
> > beeline_url = ["jdbc:hive2://{fqdn}:{spark_thrift_port}/default","transportMode={spark_transport_mode}"]
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)